{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 1. Network Layering"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Previously I suggested these two layers of networks (I have renamed them here):"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.1 Sensor-space Sequence Network"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"+---------------------+ +------------+\n",
"| sensor_t1 | | motor_t0 |\n",
"+---------------------+ +------------+\n",
" ^ ^\n",
" +---------------------+\n",
" | hidden_t0 |\n",
" +---------------------+\n",
" ^ ^\n",
"+---------------------+ +------------+\n",
"| sensor_t0 | | motor_t0 |\n",
"+---------------------+ +------------+\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1.2 Hidden-space Sequence Network"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" +---------------------+ \n",
" | hidden_t1 | \n",
" +---------------------+ \n",
" ^\n",
" +-----------------+\n",
" | hidden2_t0 |\n",
" +-----------------+\n",
" ^ ^\n",
"+---------------------+ +---------------------+ \n",
"| hidden_t0 | | hidden_goal | \n",
"+---------------------+ +---------------------+ \n",
" \n",
"\n",
"Comparing these two networks, one notices some symmetry:\n",
"\n",
"* both take state and control, and produces next state\n",
" * hidden_t0 plays the role of sensor_t0 in the Hidden-space network\n",
" * hidden_goal plays the role of motor_t0 in the Hidden-space network\n",
" \n",
"This also suggests a slight variation of the Hidden-space network:\n",
"\n",
"\n",
" +---------------------+ +---------------------+ \n",
" | hidden_t1 | | hidden_goal | \n",
" +---------------------+ +---------------------+ \n",
" ^ ^\n",
" +-----------------+\n",
" | hidden2_t0 |\n",
" +-----------------+\n",
" ^ ^\n",
"+---------------------+ +---------------------+ \n",
"| hidden_t0 | | hidden_goal | \n",
"+---------------------+ +---------------------+ \n",
" \n",
"\n",
"What does that give you?\n",
"\n",
"* complete symmetry (who cares?)\n",
"* the possibility of doing what we did with Sensor-space Network to the Hidden-space Network\n",
" * that would give us the possibility of sequencing goals\n",
" * and that is repeatable (eg, more networks doing higher level planning)\n",
"\n",
"Note: to use at a higher-level Hidden-Hidden-Space Sequence Network we will need a goal-no-op to get started."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 2. Training"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from IPython.display import display, HTML\n",
"import theano.tensor as T\n",
"import numpy as np\n",
"import sys\n",
"sys.path.append(\"..\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
]
}
],
"source": [
"from discover import *"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We import the discover program and load the experiment we are interested in:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"FLAGS.directory = \"../results\" # Where to save/restore files\n",
"FLAGS.mode = 'test' # Should be 'wander' or 'test'\n",
"FLAGS.num_steps = 5000 # Number of steps to wander and learn\n",
"FLAGS.num_hiddens = 10 # Number of hidden units in model"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"____________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"====================================================================================================\n",
"g_in (InputLayer) (None, 10) 0 \n",
"____________________________________________________________________________________________________\n",
"s_in (InputLayer) (None, 19) 0 \n",
"____________________________________________________________________________________________________\n",
"m_in (InputLayer) (None, 2) 0 \n",
"____________________________________________________________________________________________________\n",
"c_in (InputLayer) (None, 10) 0 \n",
"____________________________________________________________________________________________________\n",
"merge_1 (Merge) (None, 41) 0 \n",
"____________________________________________________________________________________________________\n",
"h (Dense) (None, 10) 420 \n",
"____________________________________________________________________________________________________\n",
"g_out (Dense) (None, 10) 110 \n",
"____________________________________________________________________________________________________\n",
"s_out (Dense) (None, 19) 209 \n",
"____________________________________________________________________________________________________\n",
"m_out (Dense) (None, 2) 22 \n",
"====================================================================================================\n",
"Total params: 761.0\n",
"Trainable params: 761.0\n",
"Non-trainable params: 0.0\n",
"____________________________________________________________________________________________________\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"../discover.py:78: UserWarning: The `merge` function is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.\n",
" x = merge([g_in, s_in, m_in, c_in], mode='concat')\n",
"/usr/local/lib/python3.5/dist-packages/keras/legacy/layers.py:456: UserWarning: The `Merge` layer is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.\n",
" name=name)\n",
"../discover.py:83: UserWarning: Update your `Model` call to the Keras 2 API: `Model(outputs=[ sensor_t0 step. That is a no-motion motor action that gives the same sensor readings (an identity function, of sorts). That we be explained more fully in the sequence model."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"goalset = [10, 29, 33, 39, 40, 59, 68]"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def build_stepwise_dataset(*goals):\n",
" \"\"\"\n",
" Given sensor[t0] + motor[t0] -> sensor[t1] + motor[t0]\n",
" \"\"\"\n",
" if len(goals) == 0:\n",
" goals = range(len(log[\"goals\"]))\n",
" data = []\n",
" for step in [log[\"goals\"][goal] for goal in goals]: # for each step goal created\n",
" # add the motor no-op\n",
" sensor_t0 = gd.history[step - gd.recall_steps]['sensors'][0]\n",
" motor = np.array([0.0, 0.0])\n",
" # identity, noop, don't move\n",
" data.append([np.concatenate([sensor_t0, (motor + 1)/2.0 ]), \n",
" np.concatenate([sensor_t0, (motor + 1)/2.0])])\n",
" for j in range(-gd.recall_steps, 2, 1):\n",
" sensor_t0 = gd.history[step + j]['sensors'][0]\n",
" motor = gd.history[step + j]['motors'][0]\n",
" sensor_t1 = gd.history[step + j + 1]['sensors'][0]\n",
" data.append([np.concatenate([sensor_t0, (motor + 1)/2.0 ]), \n",
" np.concatenate([sensor_t1, (motor + 1)/2.0])])\n",
" return data"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"motor_size = 2\n",
"sensor_size = 19\n",
"hidden_size = 25\n",
"stepwise = Network(sensor_size + motor_size, hidden_size, sensor_size + motor_size, \n",
" epsilon=0.1, momentum=0.1)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"stepwise_dataset = build_stepwise_dataset(*goalset)\n",
"stepwise.set_inputs(stepwise_dataset)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"stepwise.load(\"stepwise.net\")"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"stepwise_dict = {}\n",
"for inputs, targets in stepwise_dataset:\n",
" hidden = stepwise.layer[0].propagate(inputs)\n",
" stepwise_dict[tuple(hidden)] = targets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 5. The New Hidden-space Sequence Network"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that the single-step model is trained, we can use its hidden layer representations in the next model, the sequence network."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"hidden2_size = 50\n",
"sequence = Network(hidden_size * 2, \n",
" hidden2_size, \n",
" hidden_size + hidden_size, \n",
" epsilon=0.1, momentum=0.1) # hidden[sensors_t0, no-motor-op], fixed goal-hidden, next-hidden)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Again, because this is a feedforward network, we can build a dataset and training on each step independently. \n",
"\n",
"Note that we need a sensor + noop motor action to get started. That is, we know what our sensors are, but because we need an initial hidden-layer representation, we use a motor-noop (the don't move motor action)."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"def build_sequence_dataset(*goals):\n",
" \"\"\"\n",
" hidden[initial_sensor + noop_motor] + hidden[goal] -> hidden[sensor_t1 + motor2]\n",
" hidden[sensor_t0 + motor1] + hidden[goal] -> hidden[sensor_t1 + motor2]\n",
" \"\"\"\n",
" global sequence_dict\n",
" sequence_dict = {}\n",
" if len(goals) == 0:\n",
" goals = range(len(log[\"goals\"]))\n",
" data = []\n",
" for step in [log[\"goals\"][goal] for goal in goals]: # for each step goal created\n",
" # get the hidden[goal + last motor]\n",
" sensor_goal = gd.history[step + 1]['sensors'][0]\n",
" motor1 = gd.history[step + 1]['motors'][0]\n",
" hidden_goal = stepwise.layer[0].propagate(np.concatenate([sensor_goal, (motor1 + 1)/2.0 ]))\n",
" # add the hidden[initial sensors + motor no-op]\n",
" initial_sensor = gd.history[step - gd.recall_steps]['sensors'][0]\n",
" noop_motor = np.array([0.0, 0.0])\n",
" hidden_noop = stepwise.layer[0].propagate(np.concatenate([initial_sensor, (noop_motor + 1)/2.0 ]))\n",
" # First step:\n",
" sensor_t0 = gd.history[step - gd.recall_steps]['sensors'][0]\n",
" motor_t0 = gd.history[step - gd.recall_steps]['motors'][0]\n",
" hidden_t0 = stepwise.layer[0].propagate(np.concatenate([sensor_t0, (motor_t0 + 1)/2.0 ]))\n",
" # learn on that:\n",
" data.append([np.concatenate([hidden_noop, hidden_goal]), \n",
" np.concatenate([hidden_t0, hidden_goal])])\n",
" sequence_dict[tuple(data[-1][0])] = data[-1][1]\n",
" # now, start sequence:\n",
" for j in range(-gd.recall_steps, 1, 1):\n",
" # next hidden, motor:\n",
" sensor_t1 = gd.history[step + j + 1]['sensors'][0]\n",
" motor_t1 = gd.history[step + j + 1]['motors'][0]\n",
" hidden_t1 = stepwise.layer[0].propagate(np.concatenate([sensor_t1, (motor_t1 + 1)/2.0 ]))\n",
" data.append([np.concatenate([hidden_t0, hidden_goal]), \n",
" np.concatenate([hidden_t1, hidden_goal])])\n",
" sequence_dict[tuple(data[-1][0])] = data[-1][1]\n",
" hidden_t0 = hidden_t1\n",
" if list(sensor_goal) != list(sensor_t1) or list(hidden_goal) != list(hidden_t1):\n",
" print(\"last step is not goal!\")\n",
" stepwise.pp(\"hiden_t0 :\", hidden_t0)\n",
" stepwise.pp(\"hiden_goal:\", hidden_goal)\n",
" break\n",
" return data"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sequence_dataset = build_sequence_dataset(*goalset)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Total training input/target pairs:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"84"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(sequence_dataset)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can train on some, or all patterns. For this experiment, I trained on one, then two, then a few more, then all."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sequence.set_inputs(sequence_dataset)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------------\n",
"Training for max trails: 5000 ...\n",
"Epoch: 0 TSS error: 814.633727742 %correct: 0.0\n",
"Epoch: 5 TSS error: 56.8148474386 %correct: 0.0\n",
"Epoch: 10 TSS error: 36.9527065065 %correct: 0.0\n",
"Epoch: 15 TSS error: 32.7776544067 %correct: 1.1904761904761905\n",
"Epoch: 20 TSS error: 29.7060240751 %correct: 0.0\n",
"Epoch: 25 TSS error: 30.7119705735 %correct: 0.0\n",
"Epoch: 30 TSS error: 26.514936533 %correct: 1.1904761904761905\n",
"Epoch: 35 TSS error: 23.8863498698 %correct: 2.380952380952381\n",
"Epoch: 40 TSS error: 22.5074910913 %correct: 2.380952380952381\n",
"Epoch: 45 TSS error: 21.9453712831 %correct: 10.714285714285714\n",
"Epoch: 50 TSS error: 22.0101809933 %correct: 5.952380952380952\n",
"Epoch: 55 TSS error: 22.2418654038 %correct: 11.904761904761903\n",
"Epoch: 60 TSS error: 19.35311847 %correct: 5.952380952380952\n",
"Epoch: 65 TSS error: 18.8520892298 %correct: 10.714285714285714\n",
"Epoch: 70 TSS error: 19.3400949959 %correct: 9.523809523809524\n",
"Epoch: 75 TSS error: 18.0429622231 %correct: 14.285714285714285\n",
"Epoch: 80 TSS error: 19.5876307749 %correct: 11.904761904761903\n",
"Epoch: 85 TSS error: 18.5826907037 %correct: 2.380952380952381\n",
"Epoch: 90 TSS error: 17.0203572061 %correct: 10.714285714285714\n",
"Epoch: 95 TSS error: 16.6980123393 %correct: 25.0\n",
"Epoch: 100 TSS error: 16.0023272518 %correct: 17.857142857142858\n",
"Epoch: 105 TSS error: 16.0362472835 %correct: 17.857142857142858\n",
"Epoch: 110 TSS error: 15.7532090662 %correct: 19.047619047619047\n",
"Epoch: 115 TSS error: 16.536689149 %correct: 16.666666666666664\n",
"Epoch: 120 TSS error: 15.500339009 %correct: 20.238095238095237\n",
"Epoch: 125 TSS error: 15.2597672852 %correct: 19.047619047619047\n",
"Epoch: 130 TSS error: 15.0412323796 %correct: 22.61904761904762\n",
"Epoch: 135 TSS error: 15.1486840457 %correct: 15.476190476190476\n",
"Epoch: 140 TSS error: 16.1828507185 %correct: 15.476190476190476\n",
"Epoch: 145 TSS error: 14.7480713334 %correct: 21.428571428571427\n",
"Epoch: 150 TSS error: 15.6714152973 %correct: 13.095238095238097\n",
"Epoch: 155 TSS error: 14.3625495449 %correct: 21.428571428571427\n",
"Epoch: 160 TSS error: 15.0527025482 %correct: 17.857142857142858\n",
"Epoch: 165 TSS error: 14.1097791787 %correct: 21.428571428571427\n",
"Epoch: 170 TSS error: 13.4609319612 %correct: 22.61904761904762\n",
"Epoch: 175 TSS error: 18.3743780169 %correct: 5.952380952380952\n",
"Epoch: 180 TSS error: 13.4159049183 %correct: 21.428571428571427\n",
"Epoch: 185 TSS error: 14.0667816857 %correct: 19.047619047619047\n",
"Epoch: 190 TSS error: 14.6868301418 %correct: 16.666666666666664\n",
"Epoch: 195 TSS error: 14.027794537 %correct: 17.857142857142858\n",
"Epoch: 200 TSS error: 14.3740732277 %correct: 19.047619047619047\n",
"Epoch: 205 TSS error: 12.8915073804 %correct: 29.761904761904763\n",
"Epoch: 210 TSS error: 13.9055888445 %correct: 20.238095238095237\n",
"Epoch: 215 TSS error: 13.3222790058 %correct: 21.428571428571427\n",
"Epoch: 220 TSS error: 12.2623370406 %correct: 28.57142857142857\n",
"Epoch: 225 TSS error: 12.4773472631 %correct: 32.142857142857146\n",
"Epoch: 230 TSS error: 13.5878329406 %correct: 20.238095238095237\n",
"Epoch: 235 TSS error: 12.2554664454 %correct: 25.0\n",
"Epoch: 240 TSS error: 13.2791183912 %correct: 25.0\n",
"Epoch: 245 TSS error: 12.0504256045 %correct: 25.0\n",
"Epoch: 250 TSS error: 11.7635979028 %correct: 28.57142857142857\n",
"Epoch: 255 TSS error: 12.6712542123 %correct: 16.666666666666664\n",
"Epoch: 260 TSS error: 12.3738233163 %correct: 21.428571428571427\n",
"Epoch: 265 TSS error: 12.2967506182 %correct: 28.57142857142857\n",
"Epoch: 270 TSS error: 11.5758492691 %correct: 27.380952380952383\n",
"Epoch: 275 TSS error: 12.8479219482 %correct: 33.33333333333333\n",
"Epoch: 280 TSS error: 12.3451829186 %correct: 27.380952380952383\n",
"Epoch: 285 TSS error: 11.2173817387 %correct: 33.33333333333333\n",
"Epoch: 290 TSS error: 11.0677100571 %correct: 39.285714285714285\n",
"Epoch: 295 TSS error: 11.4699427315 %correct: 29.761904761904763\n",
"Epoch: 300 TSS error: 11.2664377328 %correct: 29.761904761904763\n",
"Epoch: 305 TSS error: 11.9058237616 %correct: 25.0\n",
"Epoch: 310 TSS error: 12.2888982987 %correct: 23.809523809523807\n",
"Epoch: 315 TSS error: 12.0889573703 %correct: 21.428571428571427\n",
"Epoch: 320 TSS error: 12.2970935777 %correct: 26.190476190476193\n",
"Epoch: 325 TSS error: 12.7404399396 %correct: 20.238095238095237\n",
"Epoch: 330 TSS error: 11.4386095176 %correct: 33.33333333333333\n",
"Epoch: 335 TSS error: 12.9347476693 %correct: 16.666666666666664\n",
"Epoch: 340 TSS error: 11.2743891735 %correct: 22.61904761904762\n",
"Epoch: 345 TSS error: 10.4532385156 %correct: 32.142857142857146\n",
"Epoch: 350 TSS error: 10.5228055535 %correct: 33.33333333333333\n",
"Epoch: 355 TSS error: 10.5294971201 %correct: 42.857142857142854\n",
"Epoch: 360 TSS error: 10.703196777 %correct: 27.380952380952383\n",
"Epoch: 365 TSS error: 9.98976926427 %correct: 42.857142857142854\n",
"Epoch: 370 TSS error: 10.649674421 %correct: 38.095238095238095\n",
"Epoch: 375 TSS error: 10.984942592 %correct: 35.714285714285715\n",
"Epoch: 380 TSS error: 9.70607918852 %correct: 44.047619047619044\n",
"Epoch: 385 TSS error: 11.7205887255 %correct: 21.428571428571427\n",
"Epoch: 390 TSS error: 10.0576325479 %correct: 40.476190476190474\n",
"Epoch: 395 TSS error: 10.4317218514 %correct: 35.714285714285715\n",
"Epoch: 400 TSS error: 9.73754988324 %correct: 44.047619047619044\n",
"Epoch: 405 TSS error: 11.1063155798 %correct: 35.714285714285715\n",
"Epoch: 410 TSS error: 9.58248353093 %correct: 44.047619047619044\n",
"Epoch: 415 TSS error: 11.7059729621 %correct: 28.57142857142857\n",
"Epoch: 420 TSS error: 9.59681278587 %correct: 41.66666666666667\n",
"Epoch: 425 TSS error: 9.38413839364 %correct: 45.23809523809524\n",
"Epoch: 430 TSS error: 10.3076727534 %correct: 33.33333333333333\n",
"Epoch: 435 TSS error: 10.7622303695 %correct: 28.57142857142857\n",
"Epoch: 440 TSS error: 10.0595165109 %correct: 36.904761904761905\n",
"Epoch: 445 TSS error: 9.88802660254 %correct: 38.095238095238095\n",
"Epoch: 450 TSS error: 9.11326843299 %correct: 46.42857142857143\n",
"Epoch: 455 TSS error: 9.66877453901 %correct: 42.857142857142854\n",
"Epoch: 460 TSS error: 9.55783765602 %correct: 41.66666666666667\n",
"Epoch: 465 TSS error: 9.61200773417 %correct: 40.476190476190474\n",
"Epoch: 470 TSS error: 9.43179571365 %correct: 39.285714285714285\n",
"Epoch: 475 TSS error: 9.27127511578 %correct: 39.285714285714285\n",
"Epoch: 480 TSS error: 9.63413841119 %correct: 36.904761904761905\n",
"Epoch: 485 TSS error: 9.7304314098 %correct: 40.476190476190474\n",
"Epoch: 490 TSS error: 8.89118646145 %correct: 42.857142857142854\n",
"Epoch: 495 TSS error: 8.71203602381 %correct: 45.23809523809524\n",
"Epoch: 500 TSS error: 9.31345435943 %correct: 45.23809523809524\n",
"Epoch: 505 TSS error: 8.92826825055 %correct: 47.61904761904761\n",
"Epoch: 510 TSS error: 11.6183433145 %correct: 17.857142857142858\n",
"Epoch: 515 TSS error: 8.92725186686 %correct: 42.857142857142854\n",
"Epoch: 520 TSS error: 8.63994971782 %correct: 45.23809523809524\n",
"Epoch: 525 TSS error: 9.2084409893 %correct: 40.476190476190474\n",
"Epoch: 530 TSS error: 8.86160010504 %correct: 46.42857142857143\n",
"Epoch: 535 TSS error: 9.06784717187 %correct: 44.047619047619044\n",
"Epoch: 540 TSS error: 9.18384723575 %correct: 38.095238095238095\n",
"Epoch: 545 TSS error: 8.71785756813 %correct: 48.80952380952381\n",
"Epoch: 550 TSS error: 8.53518999252 %correct: 48.80952380952381\n",
"Epoch: 555 TSS error: 9.92159210959 %correct: 26.190476190476193\n",
"Epoch: 560 TSS error: 8.61106132571 %correct: 50.0\n",
"Epoch: 565 TSS error: 8.65237126013 %correct: 47.61904761904761\n",
"Epoch: 570 TSS error: 8.42298668931 %correct: 53.57142857142857\n",
"Epoch: 575 TSS error: 8.67109590933 %correct: 44.047619047619044\n",
"Epoch: 580 TSS error: 9.68676857694 %correct: 42.857142857142854\n",
"Epoch: 585 TSS error: 9.74623774581 %correct: 38.095238095238095\n",
"Epoch: 590 TSS error: 9.44076036724 %correct: 34.523809523809526\n",
"Epoch: 595 TSS error: 8.31512130783 %correct: 51.19047619047619\n",
"Epoch: 600 TSS error: 9.17094069272 %correct: 35.714285714285715\n",
"Epoch: 605 TSS error: 8.19864737223 %correct: 54.761904761904766\n",
"Epoch: 610 TSS error: 8.47824126447 %correct: 42.857142857142854\n",
"Epoch: 615 TSS error: 8.81898365284 %correct: 50.0\n",
"Epoch: 620 TSS error: 9.17934433076 %correct: 46.42857142857143\n",
"Epoch: 625 TSS error: 7.97686357927 %correct: 51.19047619047619\n",
"Epoch: 630 TSS error: 10.2800889567 %correct: 26.190476190476193\n",
"Epoch: 635 TSS error: 8.24580343483 %correct: 50.0\n",
"Epoch: 640 TSS error: 7.7303396269 %correct: 51.19047619047619\n",
"Epoch: 645 TSS error: 11.6379392145 %correct: 17.857142857142858\n",
"Epoch: 650 TSS error: 8.34319019108 %correct: 40.476190476190474\n",
"Epoch: 655 TSS error: 8.28867719881 %correct: 44.047619047619044\n",
"Epoch: 660 TSS error: 9.50368769409 %correct: 40.476190476190474\n",
"Epoch: 665 TSS error: 11.668695671 %correct: 28.57142857142857\n",
"Epoch: 670 TSS error: 7.96564910345 %correct: 55.952380952380956\n",
"Epoch: 675 TSS error: 8.35573595844 %correct: 42.857142857142854\n",
"Epoch: 680 TSS error: 8.78403991097 %correct: 42.857142857142854\n",
"Epoch: 685 TSS error: 9.25537152446 %correct: 33.33333333333333\n",
"Epoch: 690 TSS error: 7.73385288504 %correct: 51.19047619047619\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 695 TSS error: 8.07605545906 %correct: 48.80952380952381\n",
"Epoch: 700 TSS error: 8.2656689451 %correct: 50.0\n",
"Epoch: 705 TSS error: 8.48305435163 %correct: 52.38095238095239\n",
"Epoch: 710 TSS error: 7.75197432309 %correct: 58.333333333333336\n",
"Epoch: 715 TSS error: 8.18026162379 %correct: 51.19047619047619\n",
"Epoch: 720 TSS error: 7.77350096772 %correct: 53.57142857142857\n",
"Epoch: 725 TSS error: 8.13232272679 %correct: 46.42857142857143\n",
"Epoch: 730 TSS error: 7.67821076901 %correct: 57.14285714285714\n",
"Epoch: 735 TSS error: 8.52742635269 %correct: 44.047619047619044\n",
"Epoch: 740 TSS error: 7.51726908028 %correct: 51.19047619047619\n",
"Epoch: 745 TSS error: 8.72989657999 %correct: 35.714285714285715\n",
"Epoch: 750 TSS error: 7.53896805679 %correct: 53.57142857142857\n",
"Epoch: 755 TSS error: 7.71742207165 %correct: 48.80952380952381\n",
"Epoch: 760 TSS error: 7.80365343862 %correct: 51.19047619047619\n",
"Epoch: 765 TSS error: 7.40527082607 %correct: 51.19047619047619\n",
"Epoch: 770 TSS error: 9.06823282179 %correct: 39.285714285714285\n",
"Epoch: 775 TSS error: 7.51962662331 %correct: 50.0\n",
"Epoch: 780 TSS error: 7.16082256638 %correct: 52.38095238095239\n",
"Epoch: 785 TSS error: 7.29789505138 %correct: 54.761904761904766\n",
"Epoch: 790 TSS error: 7.2222497199 %correct: 57.14285714285714\n",
"Epoch: 795 TSS error: 8.20130589125 %correct: 45.23809523809524\n",
"Epoch: 800 TSS error: 7.89884425664 %correct: 51.19047619047619\n",
"Epoch: 805 TSS error: 7.28456239212 %correct: 55.952380952380956\n",
"Epoch: 810 TSS error: 8.22711747705 %correct: 47.61904761904761\n",
"Epoch: 815 TSS error: 7.69475175019 %correct: 44.047619047619044\n",
"Epoch: 820 TSS error: 8.58087070703 %correct: 47.61904761904761\n",
"Epoch: 825 TSS error: 7.16883753014 %correct: 57.14285714285714\n",
"Epoch: 830 TSS error: 8.2013813425 %correct: 46.42857142857143\n",
"Epoch: 835 TSS error: 7.70278718266 %correct: 54.761904761904766\n",
"Epoch: 840 TSS error: 7.73437347103 %correct: 50.0\n",
"Epoch: 845 TSS error: 7.50591221521 %correct: 52.38095238095239\n",
"Epoch: 850 TSS error: 7.2422705007 %correct: 51.19047619047619\n",
"Epoch: 855 TSS error: 7.88692194622 %correct: 47.61904761904761\n",
"Epoch: 860 TSS error: 8.17773818042 %correct: 40.476190476190474\n",
"Epoch: 865 TSS error: 7.37500041595 %correct: 50.0\n",
"Epoch: 870 TSS error: 7.20182315842 %correct: 57.14285714285714\n",
"Epoch: 875 TSS error: 8.91959578671 %correct: 33.33333333333333\n",
"Epoch: 880 TSS error: 7.02423917464 %correct: 53.57142857142857\n",
"Epoch: 885 TSS error: 7.05018837089 %correct: 57.14285714285714\n",
"Epoch: 890 TSS error: 7.12723142076 %correct: 52.38095238095239\n",
"Epoch: 895 TSS error: 7.78683326773 %correct: 51.19047619047619\n",
"Epoch: 900 TSS error: 7.10869699934 %correct: 53.57142857142857\n",
"Epoch: 905 TSS error: 7.84821434073 %correct: 51.19047619047619\n",
"Epoch: 910 TSS error: 7.40474180446 %correct: 53.57142857142857\n",
"Epoch: 915 TSS error: 8.10644078548 %correct: 42.857142857142854\n",
"Epoch: 920 TSS error: 6.90785123166 %correct: 53.57142857142857\n",
"Epoch: 925 TSS error: 7.21988200387 %correct: 52.38095238095239\n",
"Epoch: 930 TSS error: 6.98281940968 %correct: 53.57142857142857\n",
"Epoch: 935 TSS error: 7.4204650012 %correct: 52.38095238095239\n",
"Epoch: 940 TSS error: 7.53739646971 %correct: 50.0\n",
"Epoch: 945 TSS error: 7.10684313576 %correct: 59.523809523809526\n",
"Epoch: 950 TSS error: 7.21365091579 %correct: 51.19047619047619\n",
"Epoch: 955 TSS error: 6.86060818019 %correct: 57.14285714285714\n",
"Epoch: 960 TSS error: 6.78783329105 %correct: 58.333333333333336\n",
"Epoch: 965 TSS error: 7.00131509137 %correct: 54.761904761904766\n",
"Epoch: 970 TSS error: 7.10963870005 %correct: 55.952380952380956\n",
"Epoch: 975 TSS error: 6.80216910603 %correct: 54.761904761904766\n",
"Epoch: 980 TSS error: 6.78711911541 %correct: 59.523809523809526\n",
"Epoch: 985 TSS error: 7.30358984985 %correct: 54.761904761904766\n",
"Epoch: 990 TSS error: 7.46637520785 %correct: 50.0\n",
"Epoch: 995 TSS error: 7.83928744817 %correct: 44.047619047619044\n",
"Epoch: 1000 TSS error: 7.23381340403 %correct: 63.095238095238095\n",
"Epoch: 1005 TSS error: 7.01871150951 %correct: 55.952380952380956\n",
"Epoch: 1010 TSS error: 6.86067140599 %correct: 58.333333333333336\n",
"Epoch: 1015 TSS error: 7.10850280319 %correct: 48.80952380952381\n",
"Epoch: 1020 TSS error: 7.58147170101 %correct: 52.38095238095239\n",
"Epoch: 1025 TSS error: 6.83737145239 %correct: 58.333333333333336\n",
"Epoch: 1030 TSS error: 6.84080237609 %correct: 54.761904761904766\n",
"Epoch: 1035 TSS error: 8.38889562903 %correct: 44.047619047619044\n",
"Epoch: 1040 TSS error: 7.71750607381 %correct: 41.66666666666667\n",
"Epoch: 1045 TSS error: 8.28305256365 %correct: 48.80952380952381\n",
"Epoch: 1050 TSS error: 6.96491706467 %correct: 58.333333333333336\n",
"Epoch: 1055 TSS error: 6.76513809826 %correct: 57.14285714285714\n",
"Epoch: 1060 TSS error: 6.80324145785 %correct: 60.71428571428571\n",
"Epoch: 1065 TSS error: 7.35978235019 %correct: 51.19047619047619\n",
"Epoch: 1070 TSS error: 7.97471717994 %correct: 44.047619047619044\n",
"Epoch: 1075 TSS error: 6.69549778089 %correct: 54.761904761904766\n",
"Epoch: 1080 TSS error: 8.20239380294 %correct: 50.0\n",
"Epoch: 1085 TSS error: 7.0495575864 %correct: 53.57142857142857\n",
"Epoch: 1090 TSS error: 7.1092189599 %correct: 50.0\n",
"Epoch: 1095 TSS error: 6.72393130039 %correct: 55.952380952380956\n",
"Epoch: 1100 TSS error: 6.98071961285 %correct: 58.333333333333336\n",
"Epoch: 1105 TSS error: 6.67571489564 %correct: 52.38095238095239\n",
"Epoch: 1110 TSS error: 7.26349436418 %correct: 50.0\n",
"Epoch: 1115 TSS error: 6.89230323881 %correct: 54.761904761904766\n",
"Epoch: 1120 TSS error: 6.83201895212 %correct: 60.71428571428571\n",
"Epoch: 1125 TSS error: 6.89760572446 %correct: 57.14285714285714\n",
"Epoch: 1130 TSS error: 6.62592451144 %correct: 59.523809523809526\n",
"Epoch: 1135 TSS error: 6.5364705975 %correct: 58.333333333333336\n",
"Epoch: 1140 TSS error: 6.4821877504 %correct: 57.14285714285714\n",
"Epoch: 1145 TSS error: 7.45848559274 %correct: 47.61904761904761\n",
"Epoch: 1150 TSS error: 6.85016795235 %correct: 51.19047619047619\n",
"Epoch: 1155 TSS error: 7.12707266031 %correct: 50.0\n",
"Epoch: 1160 TSS error: 6.66003525122 %correct: 58.333333333333336\n",
"Epoch: 1165 TSS error: 6.61546059455 %correct: 55.952380952380956\n",
"Epoch: 1170 TSS error: 6.96665793345 %correct: 52.38095238095239\n",
"Epoch: 1175 TSS error: 6.72494565055 %correct: 61.904761904761905\n",
"Epoch: 1180 TSS error: 7.49634737167 %correct: 59.523809523809526\n",
"Epoch: 1185 TSS error: 7.66644593397 %correct: 47.61904761904761\n",
"Epoch: 1190 TSS error: 7.16888786427 %correct: 52.38095238095239\n",
"Epoch: 1195 TSS error: 6.58645573708 %correct: 58.333333333333336\n",
"Epoch: 1200 TSS error: 6.68614316479 %correct: 59.523809523809526\n",
"Epoch: 1205 TSS error: 7.324184229 %correct: 57.14285714285714\n",
"Epoch: 1210 TSS error: 6.50569633718 %correct: 59.523809523809526\n",
"Epoch: 1215 TSS error: 7.45430171662 %correct: 48.80952380952381\n",
"Epoch: 1220 TSS error: 7.42863689583 %correct: 54.761904761904766\n",
"Epoch: 1225 TSS error: 6.48377287546 %correct: 57.14285714285714\n",
"Epoch: 1230 TSS error: 6.51936337441 %correct: 58.333333333333336\n",
"Epoch: 1235 TSS error: 7.09281068708 %correct: 53.57142857142857\n",
"Epoch: 1240 TSS error: 6.83902569597 %correct: 54.761904761904766\n",
"Epoch: 1245 TSS error: 7.31353478188 %correct: 53.57142857142857\n",
"Epoch: 1250 TSS error: 6.44318182891 %correct: 58.333333333333336\n",
"Epoch: 1255 TSS error: 6.38921966931 %correct: 64.28571428571429\n",
"Epoch: 1260 TSS error: 6.30709180818 %correct: 63.095238095238095\n",
"Epoch: 1265 TSS error: 6.5148829906 %correct: 64.28571428571429\n",
"Epoch: 1270 TSS error: 6.68864004347 %correct: 60.71428571428571\n",
"Epoch: 1275 TSS error: 6.7334594403 %correct: 59.523809523809526\n",
"Epoch: 1280 TSS error: 6.48605711357 %correct: 58.333333333333336\n",
"Epoch: 1285 TSS error: 6.88550802024 %correct: 61.904761904761905\n",
"Epoch: 1290 TSS error: 6.29477146451 %correct: 60.71428571428571\n",
"Epoch: 1295 TSS error: 6.18546432224 %correct: 65.47619047619048\n",
"Epoch: 1300 TSS error: 6.37292981993 %correct: 61.904761904761905\n",
"Epoch: 1305 TSS error: 6.32372131601 %correct: 63.095238095238095\n",
"Epoch: 1310 TSS error: 6.45526284106 %correct: 59.523809523809526\n",
"Epoch: 1315 TSS error: 6.18168383469 %correct: 59.523809523809526\n",
"Epoch: 1320 TSS error: 6.56672394663 %correct: 57.14285714285714\n",
"Epoch: 1325 TSS error: 6.32187836461 %correct: 61.904761904761905\n",
"Epoch: 1330 TSS error: 6.22889259171 %correct: 64.28571428571429\n",
"Epoch: 1335 TSS error: 6.44978753499 %correct: 57.14285714285714\n",
"Epoch: 1340 TSS error: 6.35083429706 %correct: 63.095238095238095\n",
"Epoch: 1345 TSS error: 6.17072965132 %correct: 57.14285714285714\n",
"Epoch: 1350 TSS error: 6.29881931754 %correct: 61.904761904761905\n",
"Epoch: 1355 TSS error: 6.79795512266 %correct: 50.0\n",
"Epoch: 1360 TSS error: 7.86770562567 %correct: 48.80952380952381\n",
"Epoch: 1365 TSS error: 6.40354652531 %correct: 60.71428571428571\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 1370 TSS error: 6.87658352491 %correct: 51.19047619047619\n",
"Epoch: 1375 TSS error: 6.1802057406 %correct: 61.904761904761905\n",
"Epoch: 1380 TSS error: 6.48105961825 %correct: 58.333333333333336\n",
"Epoch: 1385 TSS error: 6.47878410387 %correct: 58.333333333333336\n",
"Epoch: 1390 TSS error: 6.17205347812 %correct: 64.28571428571429\n",
"Epoch: 1395 TSS error: 6.80108489782 %correct: 52.38095238095239\n",
"Epoch: 1400 TSS error: 7.16991885305 %correct: 50.0\n",
"Epoch: 1405 TSS error: 6.18932713975 %correct: 63.095238095238095\n",
"Epoch: 1410 TSS error: 6.40827355461 %correct: 59.523809523809526\n",
"Epoch: 1415 TSS error: 7.68036171917 %correct: 45.23809523809524\n",
"Epoch: 1420 TSS error: 5.99534632062 %correct: 61.904761904761905\n",
"Epoch: 1425 TSS error: 6.8417815802 %correct: 58.333333333333336\n",
"Epoch: 1430 TSS error: 6.45706189689 %correct: 60.71428571428571\n",
"Epoch: 1435 TSS error: 6.16781749054 %correct: 61.904761904761905\n",
"Epoch: 1440 TSS error: 6.05424693645 %correct: 63.095238095238095\n",
"Epoch: 1445 TSS error: 6.09716943619 %correct: 63.095238095238095\n",
"Epoch: 1450 TSS error: 6.93816134475 %correct: 47.61904761904761\n",
"Epoch: 1455 TSS error: 6.6851624633 %correct: 52.38095238095239\n",
"Epoch: 1460 TSS error: 5.97947147422 %correct: 63.095238095238095\n",
"Epoch: 1465 TSS error: 6.28528594168 %correct: 59.523809523809526\n",
"Epoch: 1470 TSS error: 6.09018806415 %correct: 64.28571428571429\n",
"Epoch: 1475 TSS error: 6.03800643697 %correct: 63.095238095238095\n",
"Epoch: 1480 TSS error: 6.96428312007 %correct: 55.952380952380956\n",
"Epoch: 1485 TSS error: 6.7968847776 %correct: 54.761904761904766\n",
"Epoch: 1490 TSS error: 6.80001967847 %correct: 52.38095238095239\n",
"Epoch: 1495 TSS error: 6.29664866079 %correct: 61.904761904761905\n",
"Epoch: 1500 TSS error: 6.21988840634 %correct: 59.523809523809526\n",
"Epoch: 1505 TSS error: 6.14292838516 %correct: 64.28571428571429\n",
"Epoch: 1510 TSS error: 7.0497503755 %correct: 59.523809523809526\n",
"Epoch: 1515 TSS error: 7.09555051768 %correct: 57.14285714285714\n",
"Epoch: 1520 TSS error: 6.24847070143 %correct: 58.333333333333336\n",
"Epoch: 1525 TSS error: 6.18487738273 %correct: 60.71428571428571\n",
"Epoch: 1530 TSS error: 6.46426421707 %correct: 54.761904761904766\n",
"Epoch: 1535 TSS error: 6.05382903497 %correct: 65.47619047619048\n",
"Epoch: 1540 TSS error: 5.85201227996 %correct: 64.28571428571429\n",
"Epoch: 1545 TSS error: 5.94602746973 %correct: 63.095238095238095\n",
"Epoch: 1550 TSS error: 6.12099359795 %correct: 66.66666666666666\n",
"Epoch: 1555 TSS error: 6.46928834638 %correct: 47.61904761904761\n",
"Epoch: 1560 TSS error: 6.59183722705 %correct: 57.14285714285714\n",
"Epoch: 1565 TSS error: 6.23514690895 %correct: 65.47619047619048\n",
"Epoch: 1570 TSS error: 6.31383144634 %correct: 64.28571428571429\n",
"Epoch: 1575 TSS error: 6.83864202191 %correct: 58.333333333333336\n",
"Epoch: 1580 TSS error: 6.20136133995 %correct: 59.523809523809526\n",
"Epoch: 1585 TSS error: 5.86328976432 %correct: 67.85714285714286\n",
"Epoch: 1590 TSS error: 6.05008031896 %correct: 61.904761904761905\n",
"Epoch: 1595 TSS error: 6.72149930283 %correct: 55.952380952380956\n",
"Epoch: 1600 TSS error: 5.90426156474 %correct: 64.28571428571429\n",
"Epoch: 1605 TSS error: 6.11629120821 %correct: 65.47619047619048\n",
"Epoch: 1610 TSS error: 5.97026038829 %correct: 65.47619047619048\n",
"Epoch: 1615 TSS error: 6.08471327772 %correct: 66.66666666666666\n",
"Epoch: 1620 TSS error: 5.9077721966 %correct: 66.66666666666666\n",
"Epoch: 1625 TSS error: 6.15457688557 %correct: 59.523809523809526\n",
"Epoch: 1630 TSS error: 6.3726077683 %correct: 57.14285714285714\n",
"Epoch: 1635 TSS error: 6.27964496101 %correct: 55.952380952380956\n",
"Epoch: 1640 TSS error: 6.44782295921 %correct: 51.19047619047619\n",
"Epoch: 1645 TSS error: 5.74460757978 %correct: 64.28571428571429\n",
"Epoch: 1650 TSS error: 6.36915763213 %correct: 64.28571428571429\n",
"Epoch: 1655 TSS error: 5.78985522451 %correct: 65.47619047619048\n",
"Epoch: 1660 TSS error: 6.32045721129 %correct: 66.66666666666666\n",
"Epoch: 1665 TSS error: 6.18448036464 %correct: 61.904761904761905\n",
"Epoch: 1670 TSS error: 6.62479490944 %correct: 57.14285714285714\n",
"Epoch: 1675 TSS error: 6.47735627462 %correct: 60.71428571428571\n",
"Epoch: 1680 TSS error: 5.91435161722 %correct: 66.66666666666666\n",
"Epoch: 1685 TSS error: 6.26534615827 %correct: 58.333333333333336\n",
"Epoch: 1690 TSS error: 5.96040994946 %correct: 67.85714285714286\n",
"Epoch: 1695 TSS error: 6.2110012483 %correct: 58.333333333333336\n",
"Epoch: 1700 TSS error: 5.91261988495 %correct: 66.66666666666666\n",
"Epoch: 1705 TSS error: 6.16048096814 %correct: 63.095238095238095\n",
"Epoch: 1710 TSS error: 5.66730581987 %correct: 67.85714285714286\n",
"Epoch: 1715 TSS error: 5.70046614861 %correct: 63.095238095238095\n",
"Epoch: 1720 TSS error: 5.82344807061 %correct: 69.04761904761905\n",
"Epoch: 1725 TSS error: 5.93124898027 %correct: 61.904761904761905\n",
"Epoch: 1730 TSS error: 6.15308613696 %correct: 60.71428571428571\n",
"Epoch: 1735 TSS error: 6.15929970896 %correct: 61.904761904761905\n",
"Epoch: 1740 TSS error: 5.77149607569 %correct: 65.47619047619048\n",
"Epoch: 1745 TSS error: 6.56702772416 %correct: 60.71428571428571\n",
"Epoch: 1750 TSS error: 6.16182963628 %correct: 61.904761904761905\n",
"Epoch: 1755 TSS error: 5.7718201323 %correct: 64.28571428571429\n",
"Epoch: 1760 TSS error: 5.67660343894 %correct: 69.04761904761905\n",
"Epoch: 1765 TSS error: 5.5494996848 %correct: 67.85714285714286\n",
"Epoch: 1770 TSS error: 5.89257161154 %correct: 64.28571428571429\n",
"Epoch: 1775 TSS error: 6.02556924211 %correct: 63.095238095238095\n",
"Epoch: 1780 TSS error: 5.81068366546 %correct: 61.904761904761905\n",
"Epoch: 1785 TSS error: 6.6413049779 %correct: 51.19047619047619\n",
"Epoch: 1790 TSS error: 5.92828643534 %correct: 65.47619047619048\n",
"Epoch: 1795 TSS error: 5.93954168541 %correct: 67.85714285714286\n",
"Epoch: 1800 TSS error: 5.8005114002 %correct: 67.85714285714286\n",
"Epoch: 1805 TSS error: 5.68244481318 %correct: 66.66666666666666\n",
"Epoch: 1810 TSS error: 5.7652991489 %correct: 64.28571428571429\n",
"Epoch: 1815 TSS error: 5.72264747058 %correct: 63.095238095238095\n",
"Epoch: 1820 TSS error: 5.71741080951 %correct: 69.04761904761905\n",
"Epoch: 1825 TSS error: 7.96728101608 %correct: 46.42857142857143\n",
"Epoch: 1830 TSS error: 5.9373120572 %correct: 63.095238095238095\n",
"Epoch: 1835 TSS error: 5.65513108156 %correct: 67.85714285714286\n",
"Epoch: 1840 TSS error: 5.6386623482 %correct: 65.47619047619048\n",
"Epoch: 1845 TSS error: 6.03150985101 %correct: 64.28571428571429\n",
"Epoch: 1850 TSS error: 5.63744515001 %correct: 67.85714285714286\n",
"Epoch: 1855 TSS error: 5.7493075957 %correct: 67.85714285714286\n",
"Epoch: 1860 TSS error: 5.73896309628 %correct: 66.66666666666666\n",
"Epoch: 1865 TSS error: 5.46132530948 %correct: 65.47619047619048\n",
"Epoch: 1870 TSS error: 6.06540116408 %correct: 64.28571428571429\n",
"Epoch: 1875 TSS error: 5.74097643324 %correct: 61.904761904761905\n",
"Epoch: 1880 TSS error: 5.968860493 %correct: 66.66666666666666\n",
"Epoch: 1885 TSS error: 6.53323235406 %correct: 63.095238095238095\n",
"Epoch: 1890 TSS error: 5.62053582341 %correct: 66.66666666666666\n",
"Epoch: 1895 TSS error: 5.77238711655 %correct: 66.66666666666666\n",
"Epoch: 1900 TSS error: 7.32295572747 %correct: 40.476190476190474\n",
"Epoch: 1905 TSS error: 6.22027386683 %correct: 57.14285714285714\n",
"Epoch: 1910 TSS error: 5.88661128753 %correct: 63.095238095238095\n",
"Epoch: 1915 TSS error: 5.89848634397 %correct: 65.47619047619048\n",
"Epoch: 1920 TSS error: 5.77220171775 %correct: 71.42857142857143\n",
"Epoch: 1925 TSS error: 5.81195664308 %correct: 66.66666666666666\n",
"Epoch: 1930 TSS error: 5.83064188184 %correct: 63.095238095238095\n",
"Epoch: 1935 TSS error: 5.48861194979 %correct: 66.66666666666666\n",
"Epoch: 1940 TSS error: 5.92652465202 %correct: 65.47619047619048\n",
"Epoch: 1945 TSS error: 5.54277362667 %correct: 70.23809523809523\n",
"Epoch: 1950 TSS error: 5.47914755607 %correct: 69.04761904761905\n",
"Epoch: 1955 TSS error: 7.42876467151 %correct: 41.66666666666667\n",
"Epoch: 1960 TSS error: 5.85393848828 %correct: 67.85714285714286\n",
"Epoch: 1965 TSS error: 5.57198481843 %correct: 66.66666666666666\n",
"Epoch: 1970 TSS error: 5.7846504695 %correct: 70.23809523809523\n",
"Epoch: 1975 TSS error: 5.49284870649 %correct: 69.04761904761905\n",
"Epoch: 1980 TSS error: 6.49075999183 %correct: 57.14285714285714\n",
"Epoch: 1985 TSS error: 5.48551789403 %correct: 66.66666666666666\n",
"Epoch: 1990 TSS error: 5.98869168498 %correct: 64.28571428571429\n",
"Epoch: 1995 TSS error: 6.21803865856 %correct: 61.904761904761905\n",
"Epoch: 2000 TSS error: 6.76080931628 %correct: 54.761904761904766\n",
"Epoch: 2005 TSS error: 5.67661763199 %correct: 69.04761904761905\n",
"Epoch: 2010 TSS error: 5.64010253023 %correct: 66.66666666666666\n",
"Epoch: 2015 TSS error: 5.45119226116 %correct: 65.47619047619048\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 2020 TSS error: 5.9096438707 %correct: 61.904761904761905\n",
"Epoch: 2025 TSS error: 6.09631849336 %correct: 63.095238095238095\n",
"Epoch: 2030 TSS error: 5.91173488192 %correct: 61.904761904761905\n",
"Epoch: 2035 TSS error: 5.79093016952 %correct: 65.47619047619048\n",
"Epoch: 2040 TSS error: 5.59522915712 %correct: 69.04761904761905\n",
"Epoch: 2045 TSS error: 5.85719574663 %correct: 61.904761904761905\n",
"Epoch: 2050 TSS error: 5.46599539219 %correct: 66.66666666666666\n",
"Epoch: 2055 TSS error: 5.82219070204 %correct: 64.28571428571429\n",
"Epoch: 2060 TSS error: 5.60954514153 %correct: 65.47619047619048\n",
"Epoch: 2065 TSS error: 5.810848376 %correct: 61.904761904761905\n",
"Epoch: 2070 TSS error: 6.85359611601 %correct: 54.761904761904766\n",
"Epoch: 2075 TSS error: 5.46026034797 %correct: 67.85714285714286\n",
"Epoch: 2080 TSS error: 5.40980629262 %correct: 70.23809523809523\n",
"Epoch: 2085 TSS error: 6.09967290999 %correct: 63.095238095238095\n",
"Epoch: 2090 TSS error: 5.65830909471 %correct: 71.42857142857143\n",
"Epoch: 2095 TSS error: 6.35211763115 %correct: 58.333333333333336\n",
"Epoch: 2100 TSS error: 5.45172227573 %correct: 65.47619047619048\n",
"Epoch: 2105 TSS error: 5.55050922527 %correct: 69.04761904761905\n",
"Epoch: 2110 TSS error: 5.67728736784 %correct: 65.47619047619048\n",
"Epoch: 2115 TSS error: 5.47480422353 %correct: 70.23809523809523\n",
"Epoch: 2120 TSS error: 5.90239131079 %correct: 66.66666666666666\n",
"Epoch: 2125 TSS error: 5.64845029289 %correct: 69.04761904761905\n",
"Epoch: 2130 TSS error: 5.64711323865 %correct: 67.85714285714286\n",
"Epoch: 2135 TSS error: 5.29296470151 %correct: 72.61904761904762\n",
"Epoch: 2140 TSS error: 6.11210046315 %correct: 64.28571428571429\n",
"Epoch: 2145 TSS error: 5.95676379544 %correct: 61.904761904761905\n",
"Epoch: 2150 TSS error: 5.99488692903 %correct: 61.904761904761905\n",
"Epoch: 2155 TSS error: 5.99009439718 %correct: 61.904761904761905\n",
"Epoch: 2160 TSS error: 5.5653317178 %correct: 71.42857142857143\n",
"Epoch: 2165 TSS error: 5.32473638703 %correct: 66.66666666666666\n",
"Epoch: 2170 TSS error: 5.38847098922 %correct: 69.04761904761905\n",
"Epoch: 2175 TSS error: 5.36582723649 %correct: 69.04761904761905\n",
"Epoch: 2180 TSS error: 5.40826218797 %correct: 70.23809523809523\n",
"Epoch: 2185 TSS error: 5.61283917606 %correct: 66.66666666666666\n",
"Epoch: 2190 TSS error: 6.17517450086 %correct: 59.523809523809526\n",
"Epoch: 2195 TSS error: 5.50815846385 %correct: 66.66666666666666\n",
"Epoch: 2200 TSS error: 5.76494705124 %correct: 64.28571428571429\n",
"Epoch: 2205 TSS error: 5.35488768867 %correct: 69.04761904761905\n",
"Epoch: 2210 TSS error: 5.35519119223 %correct: 67.85714285714286\n",
"Epoch: 2215 TSS error: 5.73830780075 %correct: 66.66666666666666\n",
"Epoch: 2220 TSS error: 5.61087237452 %correct: 70.23809523809523\n",
"Epoch: 2225 TSS error: 5.36409094733 %correct: 70.23809523809523\n",
"Epoch: 2230 TSS error: 5.52469400528 %correct: 67.85714285714286\n",
"Epoch: 2235 TSS error: 5.47329936599 %correct: 69.04761904761905\n",
"Epoch: 2240 TSS error: 5.95712571505 %correct: 59.523809523809526\n",
"Epoch: 2245 TSS error: 6.00339031513 %correct: 65.47619047619048\n",
"Epoch: 2250 TSS error: 5.90382343132 %correct: 58.333333333333336\n",
"Epoch: 2255 TSS error: 5.7855048822 %correct: 61.904761904761905\n",
"Epoch: 2260 TSS error: 5.94588447334 %correct: 63.095238095238095\n",
"Epoch: 2265 TSS error: 5.30997664571 %correct: 70.23809523809523\n",
"Epoch: 2270 TSS error: 5.17879773147 %correct: 73.80952380952381\n",
"Epoch: 2275 TSS error: 5.39732587491 %correct: 69.04761904761905\n",
"Epoch: 2280 TSS error: 5.66033282457 %correct: 67.85714285714286\n",
"Epoch: 2285 TSS error: 5.63183465844 %correct: 67.85714285714286\n",
"Epoch: 2290 TSS error: 5.2495141541 %correct: 71.42857142857143\n",
"Epoch: 2295 TSS error: 5.12064590424 %correct: 72.61904761904762\n",
"Epoch: 2300 TSS error: 5.22276575666 %correct: 72.61904761904762\n",
"Epoch: 2305 TSS error: 5.24904957604 %correct: 71.42857142857143\n",
"Epoch: 2310 TSS error: 5.40186221156 %correct: 67.85714285714286\n",
"Epoch: 2315 TSS error: 6.13920348849 %correct: 63.095238095238095\n",
"Epoch: 2320 TSS error: 5.20950035361 %correct: 70.23809523809523\n",
"Epoch: 2325 TSS error: 6.34599941407 %correct: 61.904761904761905\n",
"Epoch: 2330 TSS error: 5.4732724034 %correct: 64.28571428571429\n",
"Epoch: 2335 TSS error: 5.25192152655 %correct: 71.42857142857143\n",
"Epoch: 2340 TSS error: 5.32443137445 %correct: 65.47619047619048\n",
"Epoch: 2345 TSS error: 5.45099707655 %correct: 65.47619047619048\n",
"Epoch: 2350 TSS error: 5.79471726545 %correct: 61.904761904761905\n",
"Epoch: 2355 TSS error: 5.29650307668 %correct: 67.85714285714286\n",
"Epoch: 2360 TSS error: 5.2482095865 %correct: 71.42857142857143\n",
"Epoch: 2365 TSS error: 5.70589852234 %correct: 69.04761904761905\n",
"Epoch: 2370 TSS error: 5.2510866405 %correct: 71.42857142857143\n",
"Epoch: 2375 TSS error: 5.28903174875 %correct: 69.04761904761905\n",
"Epoch: 2380 TSS error: 5.34093757881 %correct: 67.85714285714286\n",
"Epoch: 2385 TSS error: 5.11454225656 %correct: 70.23809523809523\n",
"Epoch: 2390 TSS error: 5.49773795849 %correct: 67.85714285714286\n",
"Epoch: 2395 TSS error: 6.03254498037 %correct: 57.14285714285714\n",
"Epoch: 2400 TSS error: 5.26463458938 %correct: 67.85714285714286\n",
"Epoch: 2405 TSS error: 5.39156893645 %correct: 70.23809523809523\n",
"Epoch: 2410 TSS error: 5.91212656426 %correct: 64.28571428571429\n",
"Epoch: 2415 TSS error: 5.17483530148 %correct: 71.42857142857143\n",
"Epoch: 2420 TSS error: 5.4736109626 %correct: 66.66666666666666\n",
"Epoch: 2425 TSS error: 5.42750172822 %correct: 69.04761904761905\n",
"Epoch: 2430 TSS error: 5.59369689765 %correct: 64.28571428571429\n",
"Epoch: 2435 TSS error: 5.86739782404 %correct: 63.095238095238095\n",
"Epoch: 2440 TSS error: 5.17611014962 %correct: 70.23809523809523\n",
"Epoch: 2445 TSS error: 5.65933240477 %correct: 64.28571428571429\n",
"Epoch: 2450 TSS error: 5.64510543331 %correct: 64.28571428571429\n",
"Epoch: 2455 TSS error: 5.53103548707 %correct: 66.66666666666666\n",
"Epoch: 2460 TSS error: 5.47402786659 %correct: 69.04761904761905\n",
"Epoch: 2465 TSS error: 5.06734857509 %correct: 72.61904761904762\n",
"Epoch: 2470 TSS error: 5.44789553453 %correct: 69.04761904761905\n",
"Epoch: 2475 TSS error: 6.41185123467 %correct: 51.19047619047619\n",
"Epoch: 2480 TSS error: 5.17883063773 %correct: 70.23809523809523\n",
"Epoch: 2485 TSS error: 5.54117902638 %correct: 63.095238095238095\n",
"Epoch: 2490 TSS error: 6.11473733839 %correct: 61.904761904761905\n",
"Epoch: 2495 TSS error: 5.15408157971 %correct: 72.61904761904762\n",
"Epoch: 2500 TSS error: 5.57502033169 %correct: 61.904761904761905\n",
"Epoch: 2505 TSS error: 5.25252094297 %correct: 69.04761904761905\n",
"Epoch: 2510 TSS error: 5.09718032704 %correct: 72.61904761904762\n",
"Epoch: 2515 TSS error: 5.14191994657 %correct: 71.42857142857143\n",
"Epoch: 2520 TSS error: 5.44435835064 %correct: 71.42857142857143\n",
"Epoch: 2525 TSS error: 5.24941245432 %correct: 69.04761904761905\n",
"Epoch: 2530 TSS error: 5.81477434235 %correct: 63.095238095238095\n",
"Epoch: 2535 TSS error: 5.42135019431 %correct: 64.28571428571429\n",
"Epoch: 2540 TSS error: 5.75963339191 %correct: 61.904761904761905\n",
"Epoch: 2545 TSS error: 5.21691216533 %correct: 71.42857142857143\n",
"Epoch: 2550 TSS error: 5.69730807696 %correct: 66.66666666666666\n",
"Epoch: 2555 TSS error: 5.37014905525 %correct: 70.23809523809523\n",
"Epoch: 2560 TSS error: 5.09463623105 %correct: 71.42857142857143\n",
"Epoch: 2565 TSS error: 5.65061039766 %correct: 64.28571428571429\n",
"Epoch: 2570 TSS error: 5.49861743478 %correct: 66.66666666666666\n",
"Epoch: 2575 TSS error: 5.36119329524 %correct: 69.04761904761905\n",
"Epoch: 2580 TSS error: 5.65198141443 %correct: 67.85714285714286\n",
"Epoch: 2585 TSS error: 5.94049159291 %correct: 61.904761904761905\n",
"Epoch: 2590 TSS error: 5.37301681608 %correct: 67.85714285714286\n",
"Epoch: 2595 TSS error: 5.44093259298 %correct: 69.04761904761905\n",
"Epoch: 2600 TSS error: 5.63952988515 %correct: 69.04761904761905\n",
"Epoch: 2605 TSS error: 4.94032982245 %correct: 72.61904761904762\n",
"Epoch: 2610 TSS error: 5.72866046915 %correct: 66.66666666666666\n",
"Epoch: 2615 TSS error: 5.25051104797 %correct: 72.61904761904762\n",
"Epoch: 2620 TSS error: 4.97250516113 %correct: 73.80952380952381\n",
"Epoch: 2625 TSS error: 6.16470250393 %correct: 59.523809523809526\n",
"Epoch: 2630 TSS error: 6.02574550742 %correct: 57.14285714285714\n",
"Epoch: 2635 TSS error: 5.91135162733 %correct: 61.904761904761905\n",
"Epoch: 2640 TSS error: 5.14503713967 %correct: 73.80952380952381\n",
"Epoch: 2645 TSS error: 5.85460196795 %correct: 59.523809523809526\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 2650 TSS error: 5.21646720455 %correct: 71.42857142857143\n",
"Epoch: 2655 TSS error: 5.34753837985 %correct: 70.23809523809523\n",
"Epoch: 2660 TSS error: 4.94287263058 %correct: 72.61904761904762\n",
"Epoch: 2665 TSS error: 6.16497874079 %correct: 59.523809523809526\n",
"Epoch: 2670 TSS error: 4.98070810933 %correct: 72.61904761904762\n",
"Epoch: 2675 TSS error: 5.1217783738 %correct: 70.23809523809523\n",
"Epoch: 2680 TSS error: 4.97450679612 %correct: 72.61904761904762\n",
"Epoch: 2685 TSS error: 5.88494952244 %correct: 63.095238095238095\n",
"Epoch: 2690 TSS error: 5.25371771775 %correct: 75.0\n",
"Epoch: 2695 TSS error: 5.2888222823 %correct: 70.23809523809523\n",
"Epoch: 2700 TSS error: 4.98791077785 %correct: 72.61904761904762\n",
"Epoch: 2705 TSS error: 5.04411515219 %correct: 69.04761904761905\n",
"Epoch: 2710 TSS error: 5.21828584983 %correct: 73.80952380952381\n",
"Epoch: 2715 TSS error: 6.30227120766 %correct: 58.333333333333336\n",
"Epoch: 2720 TSS error: 5.06479089639 %correct: 69.04761904761905\n",
"Epoch: 2725 TSS error: 5.03251415297 %correct: 75.0\n",
"Epoch: 2730 TSS error: 5.2141817793 %correct: 71.42857142857143\n",
"Epoch: 2735 TSS error: 5.15374359366 %correct: 73.80952380952381\n",
"Epoch: 2740 TSS error: 5.35795084029 %correct: 69.04761904761905\n",
"Epoch: 2745 TSS error: 5.87755137709 %correct: 58.333333333333336\n",
"Epoch: 2750 TSS error: 7.17452393086 %correct: 47.61904761904761\n",
"Epoch: 2755 TSS error: 5.97972230678 %correct: 64.28571428571429\n",
"Epoch: 2760 TSS error: 5.70302665047 %correct: 64.28571428571429\n",
"Epoch: 2765 TSS error: 4.87482250786 %correct: 73.80952380952381\n",
"Epoch: 2770 TSS error: 5.20128204288 %correct: 71.42857142857143\n",
"Epoch: 2775 TSS error: 4.913943818 %correct: 72.61904761904762\n",
"Epoch: 2780 TSS error: 5.11370380391 %correct: 69.04761904761905\n",
"Epoch: 2785 TSS error: 5.13546295204 %correct: 70.23809523809523\n",
"Epoch: 2790 TSS error: 4.98433307111 %correct: 72.61904761904762\n",
"Epoch: 2795 TSS error: 5.67620521901 %correct: 63.095238095238095\n",
"Epoch: 2800 TSS error: 4.99741300911 %correct: 72.61904761904762\n",
"Epoch: 2805 TSS error: 5.40050120106 %correct: 67.85714285714286\n",
"Epoch: 2810 TSS error: 5.37863232804 %correct: 69.04761904761905\n",
"Epoch: 2815 TSS error: 5.54171705748 %correct: 66.66666666666666\n",
"Epoch: 2820 TSS error: 5.16286719663 %correct: 71.42857142857143\n",
"Epoch: 2825 TSS error: 5.26766676799 %correct: 71.42857142857143\n",
"Epoch: 2830 TSS error: 4.99298603338 %correct: 75.0\n",
"Epoch: 2835 TSS error: 4.89804898269 %correct: 72.61904761904762\n",
"Epoch: 2840 TSS error: 4.9397301061 %correct: 73.80952380952381\n",
"Epoch: 2845 TSS error: 5.68586073046 %correct: 65.47619047619048\n",
"Epoch: 2850 TSS error: 5.73382132099 %correct: 60.71428571428571\n",
"Epoch: 2855 TSS error: 5.0067631617 %correct: 69.04761904761905\n",
"Epoch: 2860 TSS error: 5.57926461517 %correct: 66.66666666666666\n",
"Epoch: 2865 TSS error: 5.36547985127 %correct: 73.80952380952381\n",
"Epoch: 2870 TSS error: 4.88982702922 %correct: 76.19047619047619\n",
"Epoch: 2875 TSS error: 5.46089148458 %correct: 69.04761904761905\n",
"Epoch: 2880 TSS error: 5.66683534813 %correct: 66.66666666666666\n",
"Epoch: 2885 TSS error: 5.18235091425 %correct: 71.42857142857143\n",
"Epoch: 2890 TSS error: 5.55692103232 %correct: 65.47619047619048\n",
"Epoch: 2895 TSS error: 5.3525661262 %correct: 65.47619047619048\n",
"Epoch: 2900 TSS error: 6.78914326538 %correct: 51.19047619047619\n",
"Epoch: 2905 TSS error: 4.89274949834 %correct: 72.61904761904762\n",
"Epoch: 2910 TSS error: 5.16363196908 %correct: 70.23809523809523\n",
"Epoch: 2915 TSS error: 5.18819092632 %correct: 72.61904761904762\n",
"Epoch: 2920 TSS error: 5.62339405831 %correct: 66.66666666666666\n",
"Epoch: 2925 TSS error: 5.81129130657 %correct: 63.095238095238095\n",
"Epoch: 2930 TSS error: 5.07036847968 %correct: 70.23809523809523\n",
"Epoch: 2935 TSS error: 4.97238206992 %correct: 73.80952380952381\n",
"Epoch: 2940 TSS error: 4.86928418134 %correct: 71.42857142857143\n",
"Epoch: 2945 TSS error: 4.90614471778 %correct: 72.61904761904762\n",
"Epoch: 2950 TSS error: 5.36896634551 %correct: 66.66666666666666\n",
"Epoch: 2955 TSS error: 5.99437986734 %correct: 60.71428571428571\n",
"Epoch: 2960 TSS error: 4.88267964902 %correct: 72.61904761904762\n",
"Epoch: 2965 TSS error: 5.16032244782 %correct: 71.42857142857143\n",
"Epoch: 2970 TSS error: 5.60559112314 %correct: 65.47619047619048\n",
"Epoch: 2975 TSS error: 5.03420914959 %correct: 75.0\n",
"Epoch: 2980 TSS error: 5.10774300217 %correct: 72.61904761904762\n",
"Epoch: 2985 TSS error: 5.43775669766 %correct: 71.42857142857143\n",
"Epoch: 2990 TSS error: 4.83724271457 %correct: 73.80952380952381\n",
"Epoch: 2995 TSS error: 5.63236950929 %correct: 64.28571428571429\n",
"Epoch: 3000 TSS error: 6.08324703841 %correct: 63.095238095238095\n",
"Epoch: 3005 TSS error: 5.44112092227 %correct: 66.66666666666666\n",
"Epoch: 3010 TSS error: 5.7877603586 %correct: 57.14285714285714\n",
"Epoch: 3015 TSS error: 4.85104527941 %correct: 75.0\n",
"Epoch: 3020 TSS error: 5.12104289935 %correct: 69.04761904761905\n",
"Epoch: 3025 TSS error: 4.8400443369 %correct: 70.23809523809523\n",
"Epoch: 3030 TSS error: 4.84086568672 %correct: 72.61904761904762\n",
"Epoch: 3035 TSS error: 4.77996814468 %correct: 72.61904761904762\n",
"Epoch: 3040 TSS error: 4.81737902495 %correct: 75.0\n",
"Epoch: 3045 TSS error: 5.41567671466 %correct: 66.66666666666666\n",
"Epoch: 3050 TSS error: 4.89449470172 %correct: 73.80952380952381\n",
"Epoch: 3055 TSS error: 4.84608538562 %correct: 76.19047619047619\n",
"Epoch: 3060 TSS error: 5.99082832094 %correct: 58.333333333333336\n",
"Epoch: 3065 TSS error: 4.85876770404 %correct: 73.80952380952381\n",
"Epoch: 3070 TSS error: 5.56896868622 %correct: 66.66666666666666\n",
"Epoch: 3075 TSS error: 4.83062000026 %correct: 75.0\n",
"Epoch: 3080 TSS error: 4.70530439656 %correct: 75.0\n",
"Epoch: 3085 TSS error: 4.78809228188 %correct: 75.0\n",
"Epoch: 3090 TSS error: 5.49184311433 %correct: 65.47619047619048\n",
"Epoch: 3095 TSS error: 4.75366729821 %correct: 73.80952380952381\n",
"Epoch: 3100 TSS error: 5.45399912601 %correct: 66.66666666666666\n",
"Epoch: 3105 TSS error: 4.83159230362 %correct: 72.61904761904762\n",
"Epoch: 3110 TSS error: 4.82147490821 %correct: 70.23809523809523\n",
"Epoch: 3115 TSS error: 5.0326849992 %correct: 76.19047619047619\n",
"Epoch: 3120 TSS error: 5.46648722384 %correct: 70.23809523809523\n",
"Epoch: 3125 TSS error: 5.85854077413 %correct: 60.71428571428571\n",
"Epoch: 3130 TSS error: 4.87114207001 %correct: 71.42857142857143\n",
"Epoch: 3135 TSS error: 5.26460779979 %correct: 66.66666666666666\n",
"Epoch: 3140 TSS error: 4.73054859957 %correct: 73.80952380952381\n",
"Epoch: 3145 TSS error: 5.081667994 %correct: 71.42857142857143\n",
"Epoch: 3150 TSS error: 5.20669130817 %correct: 69.04761904761905\n",
"Epoch: 3155 TSS error: 5.00466435118 %correct: 72.61904761904762\n",
"Epoch: 3160 TSS error: 5.026843401 %correct: 72.61904761904762\n",
"Epoch: 3165 TSS error: 4.94412525407 %correct: 71.42857142857143\n",
"Epoch: 3170 TSS error: 4.69101048717 %correct: 73.80952380952381\n",
"Epoch: 3175 TSS error: 5.05220260906 %correct: 72.61904761904762\n",
"Epoch: 3180 TSS error: 4.99272216745 %correct: 71.42857142857143\n",
"Epoch: 3185 TSS error: 5.42647164039 %correct: 67.85714285714286\n",
"Epoch: 3190 TSS error: 4.8733026353 %correct: 72.61904761904762\n",
"Epoch: 3195 TSS error: 4.64454425096 %correct: 73.80952380952381\n",
"Epoch: 3200 TSS error: 5.02407446469 %correct: 69.04761904761905\n",
"Epoch: 3205 TSS error: 5.18726330564 %correct: 69.04761904761905\n",
"Epoch: 3210 TSS error: 5.68774096573 %correct: 61.904761904761905\n",
"Epoch: 3215 TSS error: 5.85548074468 %correct: 63.095238095238095\n",
"Epoch: 3220 TSS error: 5.29612549577 %correct: 67.85714285714286\n",
"Epoch: 3225 TSS error: 4.82741041132 %correct: 76.19047619047619\n",
"Epoch: 3230 TSS error: 4.88038035978 %correct: 72.61904761904762\n",
"Epoch: 3235 TSS error: 5.36014168845 %correct: 70.23809523809523\n",
"Epoch: 3240 TSS error: 5.08994334023 %correct: 64.28571428571429\n",
"Epoch: 3245 TSS error: 4.80752985693 %correct: 71.42857142857143\n",
"Epoch: 3250 TSS error: 4.66581275323 %correct: 71.42857142857143\n",
"Epoch: 3255 TSS error: 4.78156411277 %correct: 72.61904761904762\n",
"Epoch: 3260 TSS error: 5.39699471414 %correct: 70.23809523809523\n",
"Epoch: 3265 TSS error: 4.7103844612 %correct: 72.61904761904762\n",
"Epoch: 3270 TSS error: 4.71527414852 %correct: 75.0\n",
"Epoch: 3275 TSS error: 4.61599160482 %correct: 73.80952380952381\n",
"Epoch: 3280 TSS error: 5.50426395338 %correct: 64.28571428571429\n",
"Epoch: 3285 TSS error: 4.79175273918 %correct: 71.42857142857143\n",
"Epoch: 3290 TSS error: 4.90684320512 %correct: 72.61904761904762\n",
"Epoch: 3295 TSS error: 4.94827405114 %correct: 76.19047619047619\n",
"Epoch: 3300 TSS error: 5.53062561295 %correct: 67.85714285714286\n",
"Epoch: 3305 TSS error: 4.7504089098 %correct: 73.80952380952381\n",
"Epoch: 3310 TSS error: 4.79279691349 %correct: 75.0\n",
"Epoch: 3315 TSS error: 4.67448507099 %correct: 76.19047619047619\n",
"Epoch: 3320 TSS error: 5.07756174052 %correct: 66.66666666666666\n",
"Epoch: 3325 TSS error: 5.21417174555 %correct: 72.61904761904762\n",
"Epoch: 3330 TSS error: 4.82587401696 %correct: 77.38095238095238\n",
"Epoch: 3335 TSS error: 5.0588924472 %correct: 71.42857142857143\n",
"Epoch: 3340 TSS error: 5.08629095235 %correct: 73.80952380952381\n",
"Epoch: 3345 TSS error: 4.66943351671 %correct: 73.80952380952381\n",
"Epoch: 3350 TSS error: 4.9090139886 %correct: 70.23809523809523\n",
"Epoch: 3355 TSS error: 4.80954591282 %correct: 70.23809523809523\n",
"Epoch: 3360 TSS error: 4.97150906584 %correct: 71.42857142857143\n",
"Epoch: 3365 TSS error: 4.67653865302 %correct: 75.0\n",
"Epoch: 3370 TSS error: 4.85070758888 %correct: 72.61904761904762\n",
"Epoch: 3375 TSS error: 4.90792085475 %correct: 76.19047619047619\n",
"Epoch: 3380 TSS error: 4.81466619216 %correct: 73.80952380952381\n",
"Epoch: 3385 TSS error: 4.69490737471 %correct: 73.80952380952381\n",
"Epoch: 3390 TSS error: 4.62511625023 %correct: 71.42857142857143\n",
"Epoch: 3395 TSS error: 4.82779100485 %correct: 73.80952380952381\n",
"Epoch: 3400 TSS error: 4.7523073881 %correct: 72.61904761904762\n",
"Epoch: 3405 TSS error: 5.19919053807 %correct: 65.47619047619048\n",
"Epoch: 3410 TSS error: 4.82602747004 %correct: 75.0\n",
"Epoch: 3415 TSS error: 4.79492378842 %correct: 69.04761904761905\n",
"Epoch: 3420 TSS error: 4.70148994927 %correct: 72.61904761904762\n",
"Epoch: 3425 TSS error: 4.95011391094 %correct: 73.80952380952381\n",
"Epoch: 3430 TSS error: 5.11598196093 %correct: 75.0\n",
"Epoch: 3435 TSS error: 4.8962682335 %correct: 73.80952380952381\n",
"Epoch: 3440 TSS error: 5.13904064272 %correct: 70.23809523809523\n",
"Epoch: 3445 TSS error: 4.95741082063 %correct: 70.23809523809523\n",
"Epoch: 3450 TSS error: 4.54817337253 %correct: 75.0\n",
"Epoch: 3455 TSS error: 5.13924093842 %correct: 71.42857142857143\n",
"Epoch: 3460 TSS error: 4.8734957875 %correct: 72.61904761904762\n",
"Epoch: 3465 TSS error: 4.72631526523 %correct: 75.0\n",
"Epoch: 3470 TSS error: 4.77569535744 %correct: 75.0\n",
"Epoch: 3475 TSS error: 4.69314639991 %correct: 77.38095238095238\n",
"Epoch: 3480 TSS error: 4.67409802934 %correct: 72.61904761904762\n",
"Epoch: 3485 TSS error: 4.69148213438 %correct: 75.0\n",
"Epoch: 3490 TSS error: 4.64598200709 %correct: 76.19047619047619\n",
"Epoch: 3495 TSS error: 4.46768864009 %correct: 76.19047619047619\n",
"Epoch: 3500 TSS error: 4.55701777363 %correct: 75.0\n",
"Epoch: 3505 TSS error: 4.66278074173 %correct: 73.80952380952381\n",
"Epoch: 3510 TSS error: 4.50376663367 %correct: 76.19047619047619\n",
"Epoch: 3515 TSS error: 4.57181684683 %correct: 75.0\n",
"Epoch: 3520 TSS error: 5.21683493239 %correct: 69.04761904761905\n",
"Epoch: 3525 TSS error: 5.06819761417 %correct: 66.66666666666666\n",
"Epoch: 3530 TSS error: 4.95453931167 %correct: 70.23809523809523\n",
"Epoch: 3535 TSS error: 4.51602066387 %correct: 76.19047619047619\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 3540 TSS error: 4.95737137357 %correct: 67.85714285714286\n",
"Epoch: 3545 TSS error: 4.76843434566 %correct: 75.0\n",
"Epoch: 3550 TSS error: 4.93397314179 %correct: 73.80952380952381\n",
"Epoch: 3555 TSS error: 4.81439021825 %correct: 75.0\n",
"Epoch: 3560 TSS error: 4.96863807248 %correct: 70.23809523809523\n",
"Epoch: 3565 TSS error: 4.71910620826 %correct: 75.0\n",
"Epoch: 3570 TSS error: 4.95844974674 %correct: 66.66666666666666\n",
"Epoch: 3575 TSS error: 5.57029390155 %correct: 60.71428571428571\n",
"Epoch: 3580 TSS error: 4.73476449715 %correct: 72.61904761904762\n",
"Epoch: 3585 TSS error: 4.89014177409 %correct: 70.23809523809523\n",
"Epoch: 3590 TSS error: 5.0043055811 %correct: 70.23809523809523\n",
"Epoch: 3595 TSS error: 5.0474026913 %correct: 72.61904761904762\n",
"Epoch: 3600 TSS error: 4.58487616049 %correct: 76.19047619047619\n",
"Epoch: 3605 TSS error: 4.661190923 %correct: 76.19047619047619\n",
"Epoch: 3610 TSS error: 4.87228680274 %correct: 73.80952380952381\n",
"Epoch: 3615 TSS error: 4.65345517923 %correct: 77.38095238095238\n",
"Epoch: 3620 TSS error: 4.73681353885 %correct: 72.61904761904762\n",
"Epoch: 3625 TSS error: 4.69177910527 %correct: 75.0\n",
"Epoch: 3630 TSS error: 4.55385499259 %correct: 75.0\n",
"Epoch: 3635 TSS error: 4.68993369044 %correct: 73.80952380952381\n",
"Epoch: 3640 TSS error: 4.79035948235 %correct: 72.61904761904762\n",
"Epoch: 3645 TSS error: 4.62376828251 %correct: 76.19047619047619\n",
"Epoch: 3650 TSS error: 4.54140315702 %correct: 73.80952380952381\n",
"Epoch: 3655 TSS error: 4.75221928033 %correct: 77.38095238095238\n",
"Epoch: 3660 TSS error: 4.58123628726 %correct: 77.38095238095238\n",
"Epoch: 3665 TSS error: 4.55740865092 %correct: 76.19047619047619\n",
"Epoch: 3670 TSS error: 4.8937761695 %correct: 75.0\n",
"Epoch: 3675 TSS error: 5.00604917926 %correct: 72.61904761904762\n",
"Epoch: 3680 TSS error: 4.55640947446 %correct: 75.0\n",
"Epoch: 3685 TSS error: 4.67195962429 %correct: 73.80952380952381\n",
"Epoch: 3690 TSS error: 4.57132757945 %correct: 73.80952380952381\n",
"Epoch: 3695 TSS error: 4.69237891309 %correct: 73.80952380952381\n",
"Epoch: 3700 TSS error: 4.58615884357 %correct: 73.80952380952381\n",
"Epoch: 3705 TSS error: 4.72365823494 %correct: 73.80952380952381\n",
"Epoch: 3710 TSS error: 4.60852476543 %correct: 70.23809523809523\n",
"Epoch: 3715 TSS error: 4.73841950979 %correct: 77.38095238095238\n",
"Epoch: 3720 TSS error: 4.57030160103 %correct: 76.19047619047619\n",
"Epoch: 3725 TSS error: 4.56540064413 %correct: 76.19047619047619\n",
"Epoch: 3730 TSS error: 4.79809856576 %correct: 70.23809523809523\n",
"Epoch: 3735 TSS error: 4.39838672801 %correct: 76.19047619047619\n",
"Epoch: 3740 TSS error: 5.01518961837 %correct: 70.23809523809523\n",
"Epoch: 3745 TSS error: 4.97311371109 %correct: 72.61904761904762\n",
"Epoch: 3750 TSS error: 4.39817241427 %correct: 75.0\n",
"Epoch: 3755 TSS error: 4.4747780487 %correct: 76.19047619047619\n",
"Epoch: 3760 TSS error: 4.74333717554 %correct: 71.42857142857143\n",
"Epoch: 3765 TSS error: 4.60948661688 %correct: 76.19047619047619\n",
"Epoch: 3770 TSS error: 4.59658631132 %correct: 72.61904761904762\n",
"Epoch: 3775 TSS error: 4.64660895631 %correct: 76.19047619047619\n",
"Epoch: 3780 TSS error: 4.53850267133 %correct: 72.61904761904762\n",
"Epoch: 3785 TSS error: 5.1307650327 %correct: 70.23809523809523\n",
"Epoch: 3790 TSS error: 4.81523857883 %correct: 72.61904761904762\n",
"Epoch: 3795 TSS error: 4.922080535 %correct: 75.0\n",
"Epoch: 3800 TSS error: 4.82511806108 %correct: 72.61904761904762\n",
"Epoch: 3805 TSS error: 4.73427512619 %correct: 75.0\n",
"Epoch: 3810 TSS error: 4.78416533974 %correct: 67.85714285714286\n",
"Epoch: 3815 TSS error: 4.56218246646 %correct: 72.61904761904762\n",
"Epoch: 3820 TSS error: 5.33488768324 %correct: 69.04761904761905\n",
"Epoch: 3825 TSS error: 4.65764494667 %correct: 72.61904761904762\n",
"Epoch: 3830 TSS error: 5.09304072748 %correct: 76.19047619047619\n",
"Epoch: 3835 TSS error: 4.88163124241 %correct: 76.19047619047619\n",
"Epoch: 3840 TSS error: 5.41786207021 %correct: 64.28571428571429\n",
"Epoch: 3845 TSS error: 4.34177765208 %correct: 76.19047619047619\n",
"Epoch: 3850 TSS error: 4.56086164387 %correct: 76.19047619047619\n",
"Epoch: 3855 TSS error: 4.6280404926 %correct: 70.23809523809523\n",
"Epoch: 3860 TSS error: 5.22270382532 %correct: 71.42857142857143\n",
"Epoch: 3865 TSS error: 4.51912795854 %correct: 76.19047619047619\n",
"Epoch: 3870 TSS error: 4.40580054398 %correct: 75.0\n",
"Epoch: 3875 TSS error: 4.69532756244 %correct: 77.38095238095238\n",
"Epoch: 3880 TSS error: 4.54084757656 %correct: 76.19047619047619\n",
"Epoch: 3885 TSS error: 4.58080369747 %correct: 75.0\n",
"Epoch: 3890 TSS error: 4.85273541522 %correct: 75.0\n",
"Epoch: 3895 TSS error: 4.71057715671 %correct: 72.61904761904762\n",
"Epoch: 3900 TSS error: 4.32231060465 %correct: 75.0\n",
"Epoch: 3905 TSS error: 4.98686762033 %correct: 73.80952380952381\n",
"Epoch: 3910 TSS error: 4.7670813378 %correct: 72.61904761904762\n",
"Epoch: 3915 TSS error: 4.59757640334 %correct: 75.0\n",
"Epoch: 3920 TSS error: 4.5094784579 %correct: 75.0\n",
"Epoch: 3925 TSS error: 4.43042976191 %correct: 76.19047619047619\n",
"Epoch: 3930 TSS error: 4.48520884613 %correct: 75.0\n",
"Epoch: 3935 TSS error: 4.4527143019 %correct: 77.38095238095238\n",
"Epoch: 3940 TSS error: 4.51208696456 %correct: 77.38095238095238\n",
"Epoch: 3945 TSS error: 5.29341875459 %correct: 71.42857142857143\n",
"Epoch: 3950 TSS error: 4.37519154943 %correct: 78.57142857142857\n",
"Epoch: 3955 TSS error: 4.34073966302 %correct: 76.19047619047619\n",
"Epoch: 3960 TSS error: 4.39713441344 %correct: 76.19047619047619\n",
"Epoch: 3965 TSS error: 4.8756804167 %correct: 69.04761904761905\n",
"Epoch: 3970 TSS error: 4.61433069186 %correct: 75.0\n",
"Epoch: 3975 TSS error: 4.4450375708 %correct: 77.38095238095238\n",
"Epoch: 3980 TSS error: 5.19218407132 %correct: 71.42857142857143\n",
"Epoch: 3985 TSS error: 4.65725992793 %correct: 76.19047619047619\n",
"Epoch: 3990 TSS error: 4.50637880777 %correct: 76.19047619047619\n",
"Epoch: 3995 TSS error: 4.53922428515 %correct: 76.19047619047619\n",
"Epoch: 4000 TSS error: 4.65970435305 %correct: 71.42857142857143\n",
"Epoch: 4005 TSS error: 4.39385721178 %correct: 77.38095238095238\n",
"Epoch: 4010 TSS error: 4.69339730922 %correct: 72.61904761904762\n",
"Epoch: 4015 TSS error: 4.27126262461 %correct: 76.19047619047619\n",
"Epoch: 4020 TSS error: 4.77923966906 %correct: 76.19047619047619\n",
"Epoch: 4025 TSS error: 4.48693848351 %correct: 75.0\n",
"Epoch: 4030 TSS error: 4.57014363593 %correct: 73.80952380952381\n",
"Epoch: 4035 TSS error: 4.50276187214 %correct: 75.0\n",
"Epoch: 4040 TSS error: 4.74405008165 %correct: 75.0\n",
"Epoch: 4045 TSS error: 4.53623112412 %correct: 76.19047619047619\n",
"Epoch: 4050 TSS error: 4.5340288502 %correct: 76.19047619047619\n",
"Epoch: 4055 TSS error: 4.54885087822 %correct: 76.19047619047619\n",
"Epoch: 4060 TSS error: 4.52760592859 %correct: 75.0\n",
"Epoch: 4065 TSS error: 4.33969406813 %correct: 76.19047619047619\n",
"Epoch: 4070 TSS error: 4.34730161506 %correct: 77.38095238095238\n",
"Epoch: 4075 TSS error: 4.89113307835 %correct: 70.23809523809523\n",
"Epoch: 4080 TSS error: 4.47401261085 %correct: 73.80952380952381\n",
"Epoch: 4085 TSS error: 4.86316319754 %correct: 71.42857142857143\n",
"Epoch: 4090 TSS error: 4.32125974628 %correct: 77.38095238095238\n",
"Epoch: 4095 TSS error: 4.343728855 %correct: 78.57142857142857\n",
"Epoch: 4100 TSS error: 4.54849489655 %correct: 70.23809523809523\n",
"Epoch: 4105 TSS error: 4.58637651561 %correct: 76.19047619047619\n",
"Epoch: 4110 TSS error: 4.75520481724 %correct: 67.85714285714286\n",
"Epoch: 4115 TSS error: 5.49104804024 %correct: 64.28571428571429\n",
"Epoch: 4120 TSS error: 4.85422127051 %correct: 76.19047619047619\n",
"Epoch: 4125 TSS error: 4.67034106463 %correct: 72.61904761904762\n",
"Epoch: 4130 TSS error: 4.36985214637 %correct: 75.0\n",
"Epoch: 4135 TSS error: 4.59551702651 %correct: 73.80952380952381\n",
"Epoch: 4140 TSS error: 4.50512003693 %correct: 76.19047619047619\n",
"Epoch: 4145 TSS error: 4.18730368414 %correct: 77.38095238095238\n",
"Epoch: 4150 TSS error: 4.46559959443 %correct: 76.19047619047619\n",
"Epoch: 4155 TSS error: 4.48209961685 %correct: 76.19047619047619\n",
"Epoch: 4160 TSS error: 4.40472058317 %correct: 76.19047619047619\n",
"Epoch: 4165 TSS error: 4.32501724127 %correct: 75.0\n",
"Epoch: 4170 TSS error: 4.97519029966 %correct: 75.0\n",
"Epoch: 4175 TSS error: 4.83115231375 %correct: 72.61904761904762\n",
"Epoch: 4180 TSS error: 4.82697740867 %correct: 71.42857142857143\n",
"Epoch: 4185 TSS error: 4.44679124608 %correct: 75.0\n",
"Epoch: 4190 TSS error: 4.3740814081 %correct: 77.38095238095238\n",
"Epoch: 4195 TSS error: 4.60948002901 %correct: 73.80952380952381\n",
"Epoch: 4200 TSS error: 4.77525121157 %correct: 72.61904761904762\n",
"Epoch: 4205 TSS error: 4.62208710379 %correct: 76.19047619047619\n",
"Epoch: 4210 TSS error: 4.2470642511 %correct: 75.0\n",
"Epoch: 4215 TSS error: 4.65876693002 %correct: 75.0\n",
"Epoch: 4220 TSS error: 4.41812010717 %correct: 76.19047619047619\n",
"Epoch: 4225 TSS error: 4.4232336426 %correct: 75.0\n",
"Epoch: 4230 TSS error: 4.98103192966 %correct: 72.61904761904762\n",
"Epoch: 4235 TSS error: 4.30898659737 %correct: 76.19047619047619\n",
"Epoch: 4240 TSS error: 4.37046368635 %correct: 73.80952380952381\n",
"Epoch: 4245 TSS error: 4.38689113848 %correct: 77.38095238095238\n",
"Epoch: 4250 TSS error: 4.87410806858 %correct: 76.19047619047619\n",
"Epoch: 4255 TSS error: 4.74542842602 %correct: 72.61904761904762\n",
"Epoch: 4260 TSS error: 4.30087817361 %correct: 77.38095238095238\n",
"Epoch: 4265 TSS error: 4.60120268907 %correct: 76.19047619047619\n",
"Epoch: 4270 TSS error: 4.67797553197 %correct: 72.61904761904762\n",
"Epoch: 4275 TSS error: 4.5865024165 %correct: 76.19047619047619\n",
"Epoch: 4280 TSS error: 4.24188150722 %correct: 77.38095238095238\n",
"Epoch: 4285 TSS error: 4.94544399991 %correct: 70.23809523809523\n",
"Epoch: 4290 TSS error: 4.69622384265 %correct: 72.61904761904762\n",
"Epoch: 4295 TSS error: 4.45790181182 %correct: 78.57142857142857\n",
"Epoch: 4300 TSS error: 4.31781855157 %correct: 78.57142857142857\n",
"Epoch: 4305 TSS error: 4.93182763187 %correct: 72.61904761904762\n",
"Epoch: 4310 TSS error: 4.64110345536 %correct: 72.61904761904762\n",
"Epoch: 4315 TSS error: 4.53370876101 %correct: 77.38095238095238\n",
"Epoch: 4320 TSS error: 4.33048371971 %correct: 76.19047619047619\n",
"Epoch: 4325 TSS error: 4.79913932457 %correct: 75.0\n",
"Epoch: 4330 TSS error: 4.85086916884 %correct: 69.04761904761905\n",
"Epoch: 4335 TSS error: 4.19655901493 %correct: 78.57142857142857\n",
"Epoch: 4340 TSS error: 4.39640459025 %correct: 77.38095238095238\n",
"Epoch: 4345 TSS error: 4.70939627256 %correct: 72.61904761904762\n",
"Epoch: 4350 TSS error: 4.29548039307 %correct: 77.38095238095238\n",
"Epoch: 4355 TSS error: 4.5445385664 %correct: 72.61904761904762\n",
"Epoch: 4360 TSS error: 4.55128995633 %correct: 76.19047619047619\n",
"Epoch: 4365 TSS error: 4.62921752482 %correct: 78.57142857142857\n",
"Epoch: 4370 TSS error: 4.68583930718 %correct: 77.38095238095238\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 4375 TSS error: 4.21274209437 %correct: 76.19047619047619\n",
"Epoch: 4380 TSS error: 4.25929826908 %correct: 77.38095238095238\n",
"Epoch: 4385 TSS error: 4.61352781146 %correct: 75.0\n",
"Epoch: 4390 TSS error: 4.35885085055 %correct: 72.61904761904762\n",
"Epoch: 4395 TSS error: 4.65688086554 %correct: 72.61904761904762\n",
"Epoch: 4400 TSS error: 4.26681793802 %correct: 77.38095238095238\n",
"Epoch: 4405 TSS error: 4.75559939803 %correct: 73.80952380952381\n",
"Epoch: 4410 TSS error: 4.21538273269 %correct: 77.38095238095238\n",
"Epoch: 4415 TSS error: 4.46131687327 %correct: 76.19047619047619\n",
"Epoch: 4420 TSS error: 4.20329642351 %correct: 76.19047619047619\n",
"Epoch: 4425 TSS error: 4.33226839035 %correct: 76.19047619047619\n",
"Epoch: 4430 TSS error: 4.47571140087 %correct: 76.19047619047619\n",
"Epoch: 4435 TSS error: 4.4923802325 %correct: 72.61904761904762\n",
"Epoch: 4440 TSS error: 4.87176213122 %correct: 73.80952380952381\n",
"Epoch: 4445 TSS error: 4.7511275374 %correct: 72.61904761904762\n",
"Epoch: 4450 TSS error: 4.63007028272 %correct: 75.0\n",
"Epoch: 4455 TSS error: 4.74113290899 %correct: 73.80952380952381\n",
"Epoch: 4460 TSS error: 4.63402386941 %correct: 70.23809523809523\n",
"Epoch: 4465 TSS error: 4.4914636991 %correct: 73.80952380952381\n",
"Epoch: 4470 TSS error: 4.36884554355 %correct: 77.38095238095238\n",
"Epoch: 4475 TSS error: 4.41046559876 %correct: 76.19047619047619\n",
"Epoch: 4480 TSS error: 4.41180970028 %correct: 76.19047619047619\n",
"Epoch: 4485 TSS error: 4.86169880182 %correct: 71.42857142857143\n",
"Epoch: 4490 TSS error: 4.16778618115 %correct: 78.57142857142857\n",
"Epoch: 4495 TSS error: 4.1965747296 %correct: 76.19047619047619\n",
"Epoch: 4500 TSS error: 4.08326061039 %correct: 76.19047619047619\n",
"Epoch: 4505 TSS error: 5.01350432529 %correct: 70.23809523809523\n",
"Epoch: 4510 TSS error: 4.35647368087 %correct: 77.38095238095238\n",
"Epoch: 4515 TSS error: 4.72647909952 %correct: 69.04761904761905\n",
"Epoch: 4520 TSS error: 5.10826084914 %correct: 72.61904761904762\n",
"Epoch: 4525 TSS error: 5.52521120802 %correct: 59.523809523809526\n",
"Epoch: 4530 TSS error: 4.44500691214 %correct: 72.61904761904762\n",
"Epoch: 4535 TSS error: 4.20197262732 %correct: 75.0\n",
"Epoch: 4540 TSS error: 4.42778848199 %correct: 71.42857142857143\n",
"Epoch: 4545 TSS error: 4.49054424145 %correct: 78.57142857142857\n",
"Epoch: 4550 TSS error: 4.34679342698 %correct: 76.19047619047619\n",
"Epoch: 4555 TSS error: 4.1751339258 %correct: 77.38095238095238\n",
"Epoch: 4560 TSS error: 4.33327383715 %correct: 77.38095238095238\n",
"Epoch: 4565 TSS error: 4.50971827369 %correct: 75.0\n",
"Epoch: 4570 TSS error: 4.18940557355 %correct: 78.57142857142857\n",
"Epoch: 4575 TSS error: 4.21553657808 %correct: 77.38095238095238\n",
"Epoch: 4580 TSS error: 4.15772511239 %correct: 76.19047619047619\n",
"Epoch: 4585 TSS error: 4.76476988113 %correct: 75.0\n",
"Epoch: 4590 TSS error: 4.34179919611 %correct: 77.38095238095238\n",
"Epoch: 4595 TSS error: 4.62620606457 %correct: 72.61904761904762\n",
"Epoch: 4600 TSS error: 4.57770669918 %correct: 75.0\n",
"Epoch: 4605 TSS error: 4.67332878099 %correct: 73.80952380952381\n",
"Epoch: 4610 TSS error: 5.01453884223 %correct: 70.23809523809523\n",
"Epoch: 4615 TSS error: 4.30661288975 %correct: 77.38095238095238\n",
"Epoch: 4620 TSS error: 4.3267012536 %correct: 78.57142857142857\n",
"Epoch: 4625 TSS error: 4.4397611342 %correct: 77.38095238095238\n",
"Epoch: 4630 TSS error: 4.30246651353 %correct: 76.19047619047619\n",
"Epoch: 4635 TSS error: 4.29093138743 %correct: 77.38095238095238\n",
"Epoch: 4640 TSS error: 4.42952193397 %correct: 72.61904761904762\n",
"Epoch: 4645 TSS error: 4.0399889034 %correct: 77.38095238095238\n",
"Epoch: 4650 TSS error: 4.89480799901 %correct: 67.85714285714286\n",
"Epoch: 4655 TSS error: 4.41385684748 %correct: 76.19047619047619\n",
"Epoch: 4660 TSS error: 4.37188717161 %correct: 75.0\n",
"Epoch: 4665 TSS error: 4.3209431709 %correct: 75.0\n",
"Epoch: 4670 TSS error: 4.96867809481 %correct: 70.23809523809523\n",
"Epoch: 4675 TSS error: 4.45491613997 %correct: 76.19047619047619\n",
"Epoch: 4680 TSS error: 4.07935670542 %correct: 77.38095238095238\n",
"Epoch: 4685 TSS error: 4.08981047476 %correct: 78.57142857142857\n",
"Epoch: 4690 TSS error: 4.56675377274 %correct: 75.0\n",
"Epoch: 4695 TSS error: 4.42396691397 %correct: 77.38095238095238\n",
"Epoch: 4700 TSS error: 4.18619041793 %correct: 77.38095238095238\n",
"Epoch: 4705 TSS error: 4.04327818414 %correct: 77.38095238095238\n",
"Epoch: 4710 TSS error: 4.91593034246 %correct: 73.80952380952381\n",
"Epoch: 4715 TSS error: 4.17395648598 %correct: 79.76190476190477\n",
"Epoch: 4720 TSS error: 4.30586320261 %correct: 77.38095238095238\n",
"Epoch: 4725 TSS error: 4.7479047997 %correct: 75.0\n",
"Epoch: 4730 TSS error: 4.48481593779 %correct: 77.38095238095238\n",
"Epoch: 4735 TSS error: 4.16555745274 %correct: 78.57142857142857\n",
"Epoch: 4740 TSS error: 4.04859867469 %correct: 78.57142857142857\n",
"Epoch: 4745 TSS error: 4.45092376717 %correct: 71.42857142857143\n",
"Epoch: 4750 TSS error: 4.48109124898 %correct: 75.0\n",
"Epoch: 4755 TSS error: 4.04867986801 %correct: 78.57142857142857\n",
"Epoch: 4760 TSS error: 5.07526509539 %correct: 67.85714285714286\n",
"Epoch: 4765 TSS error: 4.36803468195 %correct: 75.0\n",
"Epoch: 4770 TSS error: 4.39235236445 %correct: 77.38095238095238\n",
"Epoch: 4775 TSS error: 4.02697588219 %correct: 77.38095238095238\n",
"Epoch: 4780 TSS error: 4.13178582481 %correct: 78.57142857142857\n",
"Epoch: 4785 TSS error: 4.40018758727 %correct: 76.19047619047619\n",
"Epoch: 4790 TSS error: 4.15454457958 %correct: 77.38095238095238\n",
"Epoch: 4795 TSS error: 4.59027506478 %correct: 76.19047619047619\n",
"Epoch: 4800 TSS error: 4.07436748535 %correct: 78.57142857142857\n",
"Epoch: 4805 TSS error: 4.39564579241 %correct: 75.0\n",
"Epoch: 4810 TSS error: 4.14483955297 %correct: 76.19047619047619\n",
"Epoch: 4815 TSS error: 3.98261139724 %correct: 77.38095238095238\n",
"Epoch: 4820 TSS error: 6.02458358913 %correct: 57.14285714285714\n",
"Epoch: 4825 TSS error: 4.09527804043 %correct: 77.38095238095238\n",
"Epoch: 4830 TSS error: 4.09527305204 %correct: 78.57142857142857\n",
"Epoch: 4835 TSS error: 3.94951791837 %correct: 78.57142857142857\n",
"Epoch: 4840 TSS error: 4.2508642369 %correct: 77.38095238095238\n",
"Epoch: 4845 TSS error: 4.20398108981 %correct: 76.19047619047619\n",
"Epoch: 4850 TSS error: 4.47467743695 %correct: 73.80952380952381\n",
"Epoch: 4855 TSS error: 3.98219410872 %correct: 77.38095238095238\n",
"Epoch: 4860 TSS error: 4.28764888719 %correct: 76.19047619047619\n",
"Epoch: 4865 TSS error: 4.97293935227 %correct: 66.66666666666666\n",
"Epoch: 4870 TSS error: 4.13755519009 %correct: 79.76190476190477\n",
"Epoch: 4875 TSS error: 4.92489402905 %correct: 76.19047619047619\n",
"Epoch: 4880 TSS error: 4.45536862845 %correct: 73.80952380952381\n",
"Epoch: 4885 TSS error: 4.22818501575 %correct: 77.38095238095238\n",
"Epoch: 4890 TSS error: 4.34109703513 %correct: 76.19047619047619\n",
"Epoch: 4895 TSS error: 4.3384695436 %correct: 76.19047619047619\n",
"Epoch: 4900 TSS error: 4.12779064126 %correct: 76.19047619047619\n",
"Epoch: 4905 TSS error: 4.35206658648 %correct: 77.38095238095238\n",
"Epoch: 4910 TSS error: 4.13541552289 %correct: 78.57142857142857\n",
"Epoch: 4915 TSS error: 4.3021677217 %correct: 75.0\n",
"Epoch: 4920 TSS error: 3.99017866613 %correct: 78.57142857142857\n",
"Epoch: 4925 TSS error: 4.05085234004 %correct: 77.38095238095238\n",
"Epoch: 4930 TSS error: 4.18936530314 %correct: 78.57142857142857\n",
"Epoch: 4935 TSS error: 4.49211336948 %correct: 72.61904761904762\n",
"Epoch: 4940 TSS error: 4.26153883553 %correct: 76.19047619047619\n",
"Epoch: 4945 TSS error: 4.18103206624 %correct: 76.19047619047619\n",
"Epoch: 4950 TSS error: 4.18989480495 %correct: 77.38095238095238\n",
"Epoch: 4955 TSS error: 4.78651272054 %correct: 63.095238095238095\n",
"Epoch: 4960 TSS error: 4.09083295242 %correct: 77.38095238095238\n",
"Epoch: 4965 TSS error: 4.41328867469 %correct: 78.57142857142857\n",
"Epoch: 4970 TSS error: 4.45172783359 %correct: 76.19047619047619\n",
"Epoch: 4975 TSS error: 3.99777208747 %correct: 76.19047619047619\n",
"Epoch: 4980 TSS error: 4.12992992589 %correct: 77.38095238095238\n",
"Epoch: 4985 TSS error: 4.22065803134 %correct: 77.38095238095238\n",
"Epoch: 4990 TSS error: 3.9729922991 %correct: 78.57142857142857\n",
"Epoch: 4995 TSS error: 4.05295079395 %correct: 78.57142857142857\n",
"Epoch: 5000 TSS error: 4.26042786628 %correct: 75.0\n",
"--------------------------------------------------\n",
"Epoch: 5000 TSS error: 4.26042786628 %correct: 75.0\n"
]
}
],
"source": [
"sequence.train(report_rate=5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is a harder problem than the single-step network. \n",
"\n",
"Does it work well enough to move the robot around?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note: datasets may be shuffled, so let's rebuild:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"stepwise_dataset = build_stepwise_dataset(*goalset)\n",
"sequence_dataset = build_sequence_dataset(*goalset)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"10 0 [ 0.5 0.5] [ 0.5 0.5]\n",
"10 1 [ 0.77100438 0.68047352] [ 0.39333636 0.40273501]\n",
"10 2 [ 0.77100438 0.68047352] [ 0.46757761 0.45222217]\n",
"10 3 [ 0.38208515 0.5904439 ] [ 0.44673756 0.44244263]\n",
"10 4 [ 0.38208515 0.5904439 ] [ 0.45596821 0.45659187]\n",
"10 5 [ 0.38208515 0.5904439 ] [ 0.45104118 0.46808372]\n",
"10 6 [ 0.38208515 0.5904439 ] [ 0.44151444 0.4783772 ]\n",
"10 7 [ 0.38208515 0.5904439 ] [ 0.43041205 0.48969324]\n",
"10 8 [ 0.38208515 0.5904439 ] [ 0.41727149 0.50322638]\n",
"10 9 [ 0.37585798 0.94142052] [ 0.40172195 0.52057243]\n",
"10 10 [ 0.37585798 0.94142052] [ 0.38431697 0.54517382]\n",
"10 11 [ 0.37585798 0.94142052] [ 0.36690623 0.58379006]\n",
"Rendering simulator images...\n",
"Goal 10\n",
"Training (left), Learned (right)\n"
]
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"29 0 [ 0.5 0.5] [ 0.5 0.5]\n",
"29 1 [ 0.5520867 0.16769625] [ 0.45435133 0.09275374]\n",
"29 2 [ 0.5520867 0.16769625] [ 0.51409493 0.20855017]\n",
"29 3 [ 0.5520867 0.16769625] [ 0.57487312 0.31269688]\n",
"29 4 [ 0.07246636 0.52682716] [ 0.63941207 0.62056541]\n",
"29 5 [ 0.5520867 0.16769625] [ 0.67899269 0.45748875]\n",
"29 6 [ 0.5520867 0.16769625] [ 0.62004138 0.5445315 ]\n",
"29 7 [ 0.5520867 0.16769625] [ 0.69103839 0.52384158]\n",
"29 8 [ 0.5520867 0.16769625] [ 0.62722454 0.50729583]\n",
"29 9 [ 0.5520867 0.16769625] [ 0.67828294 0.54774527]\n",
"29 10 [ 0.77038251 0.93194393] [ 0.64315743 0.50005556]\n",
"29 11 [ 0.5520867 0.16769625] [ 0.6636294 0.54580704]\n",
"Rendering simulator images...\n",
"Goal 29\n",
"Training (left), Learned (right)\n"
]
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"33 0 [ 0.5 0.5] [ 0.5 0.5]\n",
"33 1 [ 0.11409606 0.31777896] [ 0.09995381 0.27184658]\n",
"33 2 [ 0.11409606 0.31777896] [ 0.19130536 0.48945677]\n",
"33 3 [ 0.11409606 0.31777896] [ 0.3169312 0.78134909]\n",
"33 4 [ 0.44436408 0.89046775] [ 0.50260066 0.90876242]\n",
"33 5 [ 0.44436408 0.89046775] [ 0.50892037 0.90023116]\n",
"33 6 [ 0.44436408 0.89046775] [ 0.42047922 0.83479939]\n",
"33 7 [ 0.44436408 0.89046775] [ 0.19699095 0.59928818]\n",
"33 8 [ 0.44436408 0.89046775] [ 0.12459615 0.59456604]\n",
"33 9 [ 0.44436408 0.89046775] [ 0.16039766 0.58069629]\n",
"33 10 [ 0.03022118 0.03876438] [ 0.18978931 0.48285501]\n",
"33 11 [ 0.44436408 0.89046775] [ 0.24375137 0.6317246 ]\n",
"Rendering simulator images...\n",
"Goal 33\n",
"Training (left), Learned (right)\n"
]
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"39 0 [ 0.5 0.5] [ 0.5 0.5]\n",
"39 1 [ 0.62316554 0.78899734] [ 0.61230623 0.79823339]\n",
"39 2 [ 0.62316554 0.78899734] [ 0.60315968 0.78635067]\n",
"39 3 [ 0.62316554 0.78899734] [ 0.62295915 0.82599304]\n",
"39 4 [ 0.62316554 0.78899734] [ 0.693436 0.88717461]\n",
"39 5 [ 0.62316554 0.78899734] [ 0.78448312 0.94049974]\n",
"39 6 [ 0.84265506 0.99937114] [ 0.84770109 0.96769018]\n",
"39 7 [ 0.84265506 0.99937114] [ 0.86849825 0.97680145]\n",
"39 8 [ 0.84265506 0.99937114] [ 0.90187779 0.9760923 ]\n",
"39 9 [ 0.84265506 0.99937114] [ 0.8956465 0.96900454]\n",
"39 10 [ 0.84265506 0.99937114] [ 0.79443787 0.90267541]\n",
"39 11 [ 0.84265506 0.99937114] [ 0.7342715 0.5849787]\n",
"Rendering simulator images...\n",
"Goal 39\n",
"Training (left), Learned (right)\n"
]
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"40 0 [ 0.5 0.5] [ 0.5 0.5]\n",
"40 1 [ 0.65073144 0.38975265] [ 0.62305375 0.44137096]\n",
"40 2 [ 0.65073144 0.38975265] [ 0.60812927 0.4798707 ]\n",
"40 3 [ 0.65073144 0.38975265] [ 0.58151032 0.52298586]\n",
"40 4 [ 0.65073144 0.38975265] [ 0.56761776 0.53830204]\n",
"40 5 [ 0.33272589 0.90482181] [ 0.5647565 0.53886733]\n",
"40 6 [ 0.33272589 0.90482181] [ 0.56521938 0.53391882]\n",
"40 7 [ 0.33272589 0.90482181] [ 0.56576713 0.52985618]\n",
"40 8 [ 0.33272589 0.90482181] [ 0.56493504 0.52892569]\n",
"40 9 [ 0.33272589 0.90482181] [ 0.56310031 0.52961834]\n",
"40 10 [ 0.33272589 0.90482181] [ 0.5612353 0.52996607]\n",
"40 11 [ 0.17789765 0.86387146] [ 0.55976088 0.52945103]\n",
"Rendering simulator images...\n",
"Goal 40\n",
"Training (left), Learned (right)\n"
]
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"59 0 [ 0.5 0.5] [ 0.5 0.5]\n",
"59 1 [ 0.09984668 0.17584192] [ 0.10607549 0.16119786]\n",
"59 2 [ 0.09984668 0.17584192] [ 0.1131802 0.36717298]\n",
"59 3 [ 0.09667999 0.52738884] [ 0.10398088 0.46397506]\n",
"59 4 [ 0.09667999 0.52738884] [ 0.09920747 0.53879271]\n",
"59 5 [ 0.09667999 0.52738884] [ 0.08617281 0.53160428]\n",
"59 6 [ 0.09667999 0.52738884] [ 0.17454375 0.44172291]\n",
"59 7 [ 0.09667999 0.52738884] [ 0.13850684 0.49197482]\n",
"59 8 [ 0.09667999 0.52738884] [ 0.15825477 0.45669258]\n",
"59 9 [ 0.54958553 0.33401549] [ 0.14755484 0.48518995]\n",
"59 10 [ 0.09667999 0.52738884] [ 0.15193792 0.45753723]\n",
"59 11 [ 0.54958553 0.33401549] [ 0.15206618 0.48918803]\n",
"Rendering simulator images...\n",
"Goal 59\n",
"Training (left), Learned (right)\n"
]
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"68 0 [ 0.5 0.5] [ 0.5 0.5]\n",
"68 1 [ 0.97905087 0.23206347] [ 0.94239111 0.24044641]\n",
"68 2 [ 0.97905087 0.23206347] [ 0.942881 0.24661398]\n",
"68 3 [ 0.97905087 0.23206347] [ 0.94981036 0.23773567]\n",
"68 4 [ 0.97905087 0.23206347] [ 0.95621023 0.22606542]\n",
"68 5 [ 0.97905087 0.23206347] [ 0.96689223 0.25878462]\n",
"68 6 [ 0.97905087 0.23206347] [ 0.94756427 0.62735246]\n",
"68 7 [ 0.98767049 0.73186211] [ 0.96894001 0.41272582]\n",
"68 8 [ 0.98767049 0.73186211] [ 0.92388464 0.86719675]\n",
"68 9 [ 0.98767049 0.73186211] [ 0.97706531 0.29765194]\n",
"68 10 [ 0.98767049 0.73186211] [ 0.93262447 0.92283144]\n",
"68 11 [ 0.98767049 0.73186211] [ 0.95648971 0.46395963]\n",
"Rendering simulator images...\n",
"Goal 68\n",
"Training (left), Learned (right)\n"
]
},
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Test learning:\n",
"myseq = 0\n",
"for seq in goalset:\n",
" canvas = Canvas((200, 200))\n",
" gd.robot.useTrail = True\n",
" gd.robot.display[\"trail\"] = 1\n",
" gd.robot.display[\"body\"] = 0\n",
" gd.robot.trail[:] = []\n",
" hidden_goal = sequence_dataset[myseq * 12][0][hidden_size:]\n",
" # put robot at initial pose:\n",
" pose = log[\"poses\"][log[\"goals\"][seq] - gd.recall_steps]\n",
" gd.robot.setPose(*pose)\n",
" gd.robot.stall = log[\"stalls\"][log[\"goals\"][seq] - gd.recall_steps]\n",
" # get sensors:\n",
" sensor_t0 = gd.read_sensors()[0]\n",
" # get hidden_t0:\n",
" motor_t0 = np.array([0, 0])\n",
" hidden = stepwise.layer[0].propagate(np.concatenate([sensor_t0, (motor_t0 + 1.0)/2.0]))\n",
" h1 = sequence.propagate(np.concatenate([hidden, hidden_goal]))[:hidden_size]\n",
" if list(hidden_goal) != list(sequence_dataset[myseq * 12][0][hidden_size:]):\n",
" print(\"hidden_goal is wrong wrong!\")\n",
" break \n",
" if list(hidden) != list(sequence_dataset[myseq * 12][0][:hidden_size]):\n",
" print(\"initial hidden wrong!\")\n",
" break\n",
" for i in range(len(sequence_dataset[myseq * 12:myseq * 12 + 12])):\n",
" motor_output = stepwise.layer[1].propagate(h1)[-2:]\n",
" if i == 0:\n",
" # don't really move, that should be no-op\n",
" motor_output = np.array([0.5, 0.5])\n",
" print(seq, i, stepwise_dataset[myseq * 13 + i][0][-2:], motor_output)\n",
" motor_output = motor_output * 2.0 - 1.0\n",
" gd.robot.move(*motor_output)\n",
" gd.sim.step()\n",
" sensor_t0 = gd.read_sensors()[0]\n",
" hidden = stepwise.layer[0].propagate(np.concatenate([sensor_t0, (motor_output + 1.0)/2.0]))\n",
" h1 = sequence.propagate(np.concatenate([hidden, hidden_goal]))[:hidden_size]\n",
" myseq += 1\n",
" step = log[\"goals\"][seq]\n",
" gd.sim.draw(canvas)\n",
" training_canvas = gd.render_step_behavior(step - 10, step + 2)\n",
" print(\"Goal\", seq)\n",
" print(\"Training (left), Learned (right)\")\n",
" html = HTML(\"%s %s \" % (training_canvas.render(), canvas.render()))\n",
" display(html)\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Learns as well as the last experiment. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}